Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin should retry executeLoad method if catches SocketTimeOutException #33

Closed
sakama opened this issue Feb 24, 2016 · 0 comments · Fixed by #35
Closed

Plugin should retry executeLoad method if catches SocketTimeOutException #33

sakama opened this issue Feb 24, 2016 · 0 comments · Fixed by #35

Comments

@sakama
Copy link
Contributor

sakama commented Feb 24, 2016

This stacktrace is of embulk-output-bigquery v0.1.11, but same problem may happen with recent version.

2016-02-24 00:22:16.178 +0000 [INFO] (transaction): Loaded plugin embulk-output-bigquery (0.1.11)
2016-02-24 00:22:18.713 +0000 [INFO] (task-0000): Writing file [/path/to/.000.00.csv.gz]
2016-02-24 00:22:19.083 +0000 [INFO] (task-0000): Job preparing... project:my-project dataset:my-dataset table:my-table
2016-02-24 00:22:19.093 +0000 [INFO] (task-0000): table:[my-table] will be create if not exists
2016-02-24 00:22:19.107 +0000 [INFO] (task-0000): Upload start [/path/to/.000.00.csv.gz]
2016-02-24 00:22:39.143 +0000 [ERROR] (task-0000): Read timed out
2016-02-24 00:22:39.144 +0000 [INFO] (transaction): {done:  1 / 1, running: 0}
java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
org.embulk.exec.PartialExecutionException: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
    at org.embulk.exec.BulkLoader$LoaderState.buildPartialExecuteException(BulkLoader.java:363)
    at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:572)
    at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:33)
    at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:374)
    at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:370)
    at org.embulk.spi.Exec.doWith(Exec.java:25)
    at org.embulk.exec.BulkLoader.run(BulkLoader.java:370)
    at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:180)
    at com.treasuredata.worker.calls.EmbulkResult.run(EmbulkResult.java:62)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.treasuredata.java_call.JavaCall.processRequest(JavaCall.java:147)
    at com.treasuredata.java_call.JavaCall.run(JavaCall.java:111)
    at com.treasuredata.worker.JavaCallMain.main(JavaCallMain.java:21)
Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
    at com.google.common.base.Throwables.propagate(Throwables.java:160)
    at org.embulk.output.BigqueryOutputPlugin$2.finish(BigqueryOutputPlugin.java:323)
    at org.embulk.standards.GzipFileEncoderPlugin$1.finish(GzipFileEncoderPlugin.java:62)
    at org.embulk.spi.util.OutputStreamFileOutput.finish(OutputStreamFileOutput.java:58)
    at org.embulk.spi.util.FileOutputOutputStream.close(FileOutputOutputStream.java:104)
    at sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:320)
    at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:149)
    at java.io.OutputStreamWriter.close(OutputStreamWriter.java:233)
    at java.io.BufferedWriter.close(BufferedWriter.java:266)
    at org.embulk.spi.util.LineEncoder.finish(LineEncoder.java:100)
    at org.embulk.standards.CsvFormatterPlugin$1.finish(CsvFormatterPlugin.java:206)
    at org.embulk.spi.FileOutputRunner$DelegateTransactionalPageOutput.finish(FileOutputRunner.java:172)
    at org.embulk.spi.PageBuilder.finish(PageBuilder.java:223)
    at org.embulk.spi.FileInputRunner.run(FileInputRunner.java:154)
    at org.embulk.spi.util.Executors.process(Executors.java:67)
    at org.embulk.spi.util.Executors.process(Executors.java:42)
    at org.embulk.exec.LocalExecutorPlugin$2.call(LocalExecutorPlugin.java:104)
    at org.embulk.exec.LocalExecutorPlugin$2.call(LocalExecutorPlugin.java:100)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
    Suppressed: java.lang.IllegalStateException: Current state = FLUSHED, new state = CODING_END
        at java.nio.charset.CharsetEncoder.throwIllegalStateException(CharsetEncoder.java:968)
        at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:554)
        at sun.nio.cs.StreamEncoder.flushLeftoverChar(StreamEncoder.java:242)
        at sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:301)
        at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:149)
        at java.io.OutputStreamWriter.close(OutputStreamWriter.java:233)
        at java.io.BufferedWriter.close(BufferedWriter.java:266)
        at org.embulk.spi.util.LineEncoder.close(LineEncoder.java:114)
        at org.embulk.standards.CsvFormatterPlugin$1.close(CsvFormatterPlugin.java:211)
        at org.embulk.spi.FileOutputRunner$DelegateTransactionalPageOutput.close(FileOutputRunner.java:178)
        at org.embulk.spi.CloseResource.close(CloseResource.java:34)
        at org.embulk.spi.util.Executors.process(Executors.java:81)
        ... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:152)
    at java.net.SocketInputStream.read(SocketInputStream.java:122)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
    at sun.security.ssl.InputRecord.read(InputRecord.java:480)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:934)
    at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:891)
    at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
    at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:689)
    at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1324)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
    at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:37)
    at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:94)
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:972)
    at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequestWithoutGZip(MediaHttpUploader.java:545)
    at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequest(MediaHttpUploader.java:562)
    at com.google.api.client.googleapis.media.MediaHttpUploader.executeUploadInitiation(MediaHttpUploader.java:519)
    at com.google.api.client.googleapis.media.MediaHttpUploader.resumableUpload(MediaHttpUploader.java:384)
    at com.google.api.client.googleapis.media.MediaHttpUploader.upload(MediaHttpUploader.java:336)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:427)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
    at org.embulk.output.BigqueryWriter.executeLoad(BigqueryWriter.java:181)
    at org.embulk.output.BigqueryOutputPlugin$2.finish(BigqueryOutputPlugin.java:315)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging a pull request may close this issue.

1 participant