{"issue":{"created_at":"2011-12-01T20:11:13Z","state":"open","updated_at":"2011-12-01T20:36:21Z","comments":2,"user":{"gravatar_id":"1f029d4837c64a5934e7b65276f7e178","id":34254,"url":"https://api.github.com/users/gregjan","avatar_url":"https://secure.gravatar.com/avatar/1f029d4837c64a5934e7b65276f7e178?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png","login":"gregjan"},"body":"This happened while ingesting the Ralph Epperson 04 SIP within the SFC. The ingest contains one especially large data file, at 4.9 gig. This is the file that is referenced in the stack trace below:\r\n\r\n    Your submission was not ingested into the repository.\r\n\r\n    Error: org.fcrepo.server.errors.LowlevelStorageException: IRODSFedoraFileSystem.write(): couldn't make directories for [/cdrZone/home/fedora/datastreams/2011/1201/10/02/uuid_5fa6361f-b7b1-490b-b2a2-4b8fc8babdf4+DATA_FILE+DATA_FILE.0] JargonException caught in constructor, rethrow as IOException; nested exception is org.springframework.ws.soap.client.SoapFaultClientException: org.fcrepo.server.errors.LowlevelStorageException: IRODSFedoraFileSystem.write(): couldn't make directories for [/cdrZone/home/fedora/datastreams/2011/1201/10/02/uuid_5fa6361f-b7b1-490b-b2a2-4b8fc8babdf4+DATA_FILE+DATA_FILE.0] JargonException caught in constructor, rethrow as IOException\r\n\r\ndigging deeper, we find this in the fedora log (in reference to the same path as above):\r\n\r\nCaused by: java.io.IOException: JargonException caught in constructor, rethrow as IOException\r\n        at org.irods.jargon.core.pub.io.IRODSFileOutputStream.close(IRODSFileOutputStream.java:140) [jargon-core-3.0.1-20111012.172326-5.jar:na]\r\n        at java.io.FilterOutputStream.close(FilterOutputStream.java:143) [na:1.6.0_29]\r\n        at fedorax.server.module.storage.lowlevel.irods.IrodsIFileSystem.stream2streamCopy(IrodsIFileSystem.java:110) [fcrepo-irods-storage-2.1.jar:na]\r\n        at fedorax.server.module.storage.lowlevel.irods.IrodsIFileSystem.write(IrodsIFileSystem.java:356) [fcrepo-irods-storage-2.1.jar:na]\r\n        ... 62 common frames omitted\r\nCaused by: org.irods.jargon.core.exception.JargonException: java.net.SocketTimeoutException: Read timed out\r\n        at org.irods.jargon.core.connection.IRODSCommands.readHeaderLength(IRODSCommands.java:954) [jargon-core-3.0.1-20111012.172326-5.jar:na]\r\n        at org.irods.jargon.core.connection.IRODSCommands.readHeader(IRODSCommands.java:825) [jargon-core-3.0.1-20111012.172326-5.jar:na]\r\n        at org.irods.jargon.core.connection.IRODSCommands.readMessage(IRODSCommands.java:636) [jargon-core-3.0.1-20111012.172326-5.jar:na]\r\n        at org.irods.jargon.core.connection.IRODSCommands.readMessage(IRODSCommands.java:624) [jargon-core-3.0.1-20111012.172326-5.jar:na]\r\n        at org.irods.jargon.core.connection.IRODSCommands.irodsFunction(IRODSCommands.java:258) [jargon-core-3.0.1-20111012.172326-5.jar:na]\r\n        at org.irods.jargon.core.connection.IRODSCommands.irodsFunction(IRODSCommands.java:192) [jargon-core-3.0.1-20111012.172326-5.jar:na]\r\n        at org.irods.jargon.core.pub.IRODSFileSystemAOImpl.fileClose(IRODSFileSystemAOImpl.java:1414) [jargon-core-3.0.1-20111012.172326-5.jar:na]\r\n        at org.irods.jargon.core.pub.io.IRODSFileImpl.close(IRODSFileImpl.java:1337) [jargon-core-3.0.1-20111012.172326-5.jar:na]\r\n        at org.irods.jargon.core.pub.io.IRODSFileOutputStream.close(IRODSFileOutputStream.java:136) [jargon-core-3.0.1-20111012.172326-5.jar:na]\r\n        ... 65 common frames omitted\r\nCaused by: java.net.SocketTimeoutException: Read timed out\r\n        at java.net.SocketInputStream.socketRead0(Native Method) [na:1.6.0_29]\r\n        at java.net.SocketInputStream.read(SocketInputStream.java:129) [na:1.6.0_29]\r\n        at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) [na:1.6.0_29]\r\n        at java.io.BufferedInputStream.read1(BufferedInputStream.java:258) [na:1.6.0_29]\r\n        at java.io.BufferedInputStream.read(BufferedInputStream.java:317) [na:1.6.0_29]\r\n","assignee":null,"id":2419151,"labels":[{"name":"incident report","color":"DDDDDD","url":"https://api.github.com/repos/UNC-Libraries/Carolina-Digital-Repository/labels/incident+report"}],"pull_request":{"patch_url":null,"html_url":null,"diff_url":null},"number":28,"title":"IRODSFedoraFileSystem.write(): couldn't make directories","closed_at":null,"html_url":"https://github.com/UNC-Libraries/Carolina-Digital-Repository/issues/28","url":"https://api.github.com/repos/UNC-Libraries/Carolina-Digital-Repository/issues/28","milestone":null},"action":"created","comment":{"created_at":"2011-12-01T20:36:21Z","updated_at":"2011-12-01T20:36:21Z","user":{"gravatar_id":"1f029d4837c64a5934e7b65276f7e178","id":34254,"url":"https://api.github.com/users/gregjan","avatar_url":"https://secure.gravatar.com/avatar/1f029d4837c64a5934e7b65276f7e178?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png","login":"gregjan"},"body":"Vault Cleanup! \r\n\r\nNote: This issue leaves a single file from each failed ingest in the vault.\r\n\r\nJargon throws an exception, which ends ingest. Fedora considers the PUT operation to iRODS to be a failure..  However, since this is just a timeout, the file actually remains in iRODS and in the vault.  So iRODS has this leftover file every time we reject an ingest due to this timeout.  The patch will fix the issue, but it will not remove these extra files in iRODS.\r\n\r\nI will compile a list of the files we need to clear from iRODS. There is 1 file per failed SIP and some are already documented in our incident history.","id":2980206,"url":"https://api.github.com/repos/UNC-Libraries/Carolina-Digital-Repository/issues/comments/2980206"}}