{"issue":{"created_at":"2011-11-16T18:24:31Z","state":"open","updated_at":"2011-11-21T17:31:16Z","user":{"gravatar_id":"8eff4f840b1605e58ce34b0c3b5eea5a","id":902700,"url":"https://api.github.com/users/jakesmith","login":"jakesmith","avatar_url":"https://secure.gravatar.com/avatar/8eff4f840b1605e58ce34b0c3b5eea5a?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png"},"comments":6,"body":"It was assumed the csv header lines, existed on the 1st node, but they might\r\nexist on any, and/or span nodes.\r\nIf a csv input has a header, it will now mean that the nodes must wait until\r\nsignalled by the previous node before proceeding. Once the header lines are\r\nconsumed, the consumer will signal all remaining nodes immediately.\r\nSuperfile with CSV's make this particularly nasty to deal with, as they may \r\nhave headers that start or span different nodes to each other.\r\n\r\nSigned-off-by: Jake Smith <jake.smith@lexisnexis.com>","assignee":null,"id":2259819,"pull_request":{"patch_url":"https://github.com/hpcc-systems/HPCC-Platform/pull/1000.patch","html_url":"https://github.com/hpcc-systems/HPCC-Platform/pull/1000","diff_url":"https://github.com/hpcc-systems/HPCC-Platform/pull/1000.diff"},"number":1000,"labels":[],"title":"Fix gh-914, Ensure CSV headers skipped","closed_at":null,"html_url":"https://github.com/hpcc-systems/HPCC-Platform/issues/1000","url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/issues/1000","milestone":null},"action":"created","comment":{"created_at":"2011-11-21T17:31:16Z","updated_at":"2011-11-21T17:31:16Z","user":{"gravatar_id":"8eff4f840b1605e58ce34b0c3b5eea5a","id":902700,"url":"https://api.github.com/users/jakesmith","login":"jakesmith","avatar_url":"https://secure.gravatar.com/avatar/8eff4f840b1605e58ce34b0c3b5eea5a?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png"},"body":"I take it back.\r\n1) 1st node is waiting for end of input before sending, even if N was consumed.\r\n2) There might be subfiles with many records and some sparse with headers skewed to end, if so - as it stands it would have to read to end-of-input for all subs, because headers not consumed for the sparse skewed one until late.","id":2818152,"url":"https://api.github.com/repos/hpcc-systems/HPCC-Platform/issues/comments/2818152"}}