What is the max JDBC batch size? [ANSWERED]

What is the max JDBC batch size?

Asked By: vlyalcin
Originally Asked On: 2014-01-02 13:22:13
Asked Via: stackoverflow

I have a list and that list increasing continuously. I am doing add batch depend on the list size. I forgot to put limit for do executeBatch in specified size.

Program is working for hours. I dont want to stop, fix and start again for now.

My questions, what decides size of the adding batch? What is the max capacity of the batch to do executeBatch() in a one time? How many time I can use addBatch without do executeBatch()?

He received 3 answers
eventually accepting:

Craig Ringer’s answer to

What is the max JDBC batch size?

PgJDBC has some limitations regarding batches:

• All request values, and all results, must be accumulated in memory. This includes large blob/clob results. So free memory is the main limiting factor for batch size.

• Until PgJDBC 9.4 (not yet released), batches that return generated keys always do a round trip for every entry, so they’re no better than individual statement executions.

• Even in 9.4, batches that return generated keys only offer a benefit if the generated values are size limited. A single text, bytea or unconstrained varchar field in the requested result will force the driver to do a round trip for every execution.

The benefit of batching is a reduction in network round trips. So there’s much less point if your DB is local to your app server. There’s a diminishing return with increasing batch size, because the total time taken in network waits falls off quickly, so it’s often not work stressing about trying to make batches as big as possible.

If you’re bulk-loading data, seriously consider using the COPY API instead, via PgJDBC’s CopyManager, obtained via the PgConnection interface. It lets you stream CSV-like data to the server for rapid bulk-loading with very few client/server round trips. Unfortunately, it’s remarkably under-documented – it doesn’t appear in the main PgJDBC docs at all, only in the API docs.

The answer with the highest score with 1 points was:

asafm’s answer to

What is the max JDBC batch size?

AFAIK there is no limit beside the memory issue.
regarding your question: the statement is sent to the DB only on execute batch so until you’ll execute the batch the memory will continue to grow until you will get JavaHeapSpace or the batch will be sent to the DB.

If the selected answer did not help you out, the other answers might!

All Answers For: What is the max JDBC batch size?

asafm’s answer to

What is the max JDBC batch size?

AFAIK there is no limit beside the memory issue.
regarding your question: the statement is sent to the DB only on execute batch so until you’ll execute the batch the memory will continue to grow until you will get JavaHeapSpace or the batch will be sent to the DB.

Craig Ringer’s answer to

What is the max JDBC batch size?

PgJDBC has some limitations regarding batches:

• All request values, and all results, must be accumulated in memory. This includes large blob/clob results. So free memory is the main limiting factor for batch size.

• Until PgJDBC 9.4 (not yet released), batches that return generated keys always do a round trip for every entry, so they’re no better than individual statement executions.

• Even in 9.4, batches that return generated keys only offer a benefit if the generated values are size limited. A single text, bytea or unconstrained varchar field in the requested result will force the driver to do a round trip for every execution.

The benefit of batching is a reduction in network round trips. So there’s much less point if your DB is local to your app server. There’s a diminishing return with increasing batch size, because the total time taken in network waits falls off quickly, so it’s often not work stressing about trying to make batches as big as possible.

If you’re bulk-loading data, seriously consider using the COPY API instead, via PgJDBC’s CopyManager, obtained via the PgConnection interface. It lets you stream CSV-like data to the server for rapid bulk-loading with very few client/server round trips. Unfortunately, it’s remarkably under-documented – it doesn’t appear in the main PgJDBC docs at all, only in the API docs.

Martin’s answer to

What is the max JDBC batch size?

There may be a maximum number of parameter markers depending on the JDBC implementation.

For instance the PostgreSQL driver represents the number of parameters as a 2-byte integer, which in Java is at most 32768.

Of course, you should really check out the original question.

The post What is the max JDBC batch size? [ANSWERED] appeared first on Tech ABC to XYZ.