Ingest: CSV ingest gets stuck in "ingest in progress" mode with non-comforming csv file

[eaquigley]

---

Author Name: Kevin Condon (@kcondon)
Original Redmine Issue: 3992, https://redmine.hmdc.harvard.edu/issues/3992
Original Date: 2014-05-19
Original Assignee: Leonid Andreev

---

Tested on 5/19

I created a test csv file that is valid except for the last row, which is missing a comma between 2 values and so appears to have 1 less column.

It is detected as csv on upload but gets stuck with the ingest in progress notice and there is a server log error:

[2014-05-19T17:41:18.261-0400] [glassfish 4.0] [SEVERE] [] [] [tid: _ThreadID=1083 _ThreadName=Thread-4] [timeMillis: 1400535678261] [levelValue: 1000] [[
java.io.IOException: Reading mismatch, line 13 of the Data file: 4 delimited values expected, 3 found.
at edu.harvard.iq.dataverse.ingest.tabulardata.impl.plugins.csv.CSVFileReader.readFile(CSVFileReader.java:228)
at edu.harvard.iq.dataverse.ingest.tabulardata.impl.plugins.csv.CSVFileReader.read(CSVFileReader.java:126)
at edu.harvard.iq.dataverse.ingest.IngestServiceBean.ingestAsTabular(IngestServiceBean.java:246)
at edu.harvard.iq.dataverse.ingest.IngestServiceBean.ingestAsTabular(IngestServiceBean.java:203)
at sun.reflect.GeneratedMethodAccessor6717.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4695)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:630)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:582)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:46)
at sun.reflect.GeneratedMethodAccessor6187.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:582)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
at sun.reflect.GeneratedMethodAccessor6188.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4667)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4655)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
at com.sun.proxy.$Proxy461.ingestAsTabular(Unknown Source)
at edu.harvard.iq.dataverse.ingest.EJB31_Generated__IngestServiceBean__Intf____Bean.ingestAsTabular(Unknown Source)
at edu.harvard.iq.dataverse.ingest.IngestMessageBean.onMessage(IngestMessageBean.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4695)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:630)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:582)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:55)
at sun.reflect.GeneratedMethodAccessor6187.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:822)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4667)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4655)
at org.glassfish.ejb.mdb.MessageBeanContainer.deliverMessage(MessageBeanContainer.java:1219)
at org.glassfish.ejb.mdb.MessageBeanListenerImpl.deliverMessage(MessageBeanListenerImpl.java:81)
at com.sun.enterprise.connectors.inbound.MessageEndpointInvocationHandler.invoke(MessageEndpointInvocationHandler.java:171)
at com.sun.proxy.$Proxy475.onMessage(Unknown Source)
at com.sun.messaging.jms.ra.OnMessageRunner.run(OnMessageRunner.java:283)
at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:107)
at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497)
at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540)]]

[landreev]

This should no longer be happening.
The "ingest failed" icon will be displayed. And the error report will specify the line in which the token number mismatch has been detected.

[kcondon]

Reported case now failing gracefully.