{"issue":{"state":"open","pull_request":{"diff_url":null,"html_url":null,"patch_url":null},"milestone":null,"user":{"avatar_url":"https://secure.gravatar.com/avatar/11c1a8701fcf16f2c8f8a6b7f519825c?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png","gravatar_id":"11c1a8701fcf16f2c8f8a6b7f519825c","url":"https://api.github.com/users/miguno","login":"miguno","id":294849},"labels":[],"updated_at":"2011-11-10T11:27:45Z","assignee":null,"closed_at":null,"comments":5,"url":"https://api.github.com/repos/kevinweil/elephant-bird/issues/65","body":"We have run into some issues with LzoJsonLoader and JsonLoader where on some JSON data -- we haven't been able to narrow it down yet -- both classes will fail with a NullPointerException.\r\n\r\nFor instance, here is the relevant snippet of LzoJsonRecordReader#decodeLineToJson().  What seems to happen is that LzoJsonRecordReader#nextKeyValue() passes a null value to decodeLineToJson().  The problem might therefore originate in the line \"int newSize = in_.readLine(currentLine_);\" in nextKeyValue().  From what I remember, json-simple's parse method might actually return a null value as well (= adding a safeguard to LzoJsonLoader and JsonLoader might be a good idea anyways) but in this case the null value is already passed to json-simple, before the latter will actually return itself with a null value.\r\n\r\nA similar NPE issue exists for the JsonLoader#parseStringToTuple().\r\n\r\n```\r\n/*\r\n    LzoJsonRecordReader.java\r\n*/\r\n \r\n public static boolean decodeLineToJson(JSONParser parser, Text line, MapWritable value) {\r\n    try {\r\n      JSONObject jsonObj = (JSONObject)parser.parse(line.toString());\r\n      for (Object key: jsonObj.keySet()) {      /* *** The NPE is thrown here when jsonObj == null *** */\r\n        Text mapKey = new Text(key.toString());\r\n        Text mapValue = new Text();\r\n        if (jsonObj.get(key) != null) {\r\n          mapValue.set(jsonObj.get(key).toString());\r\n        }\r\n\r\n        value.put(mapKey, mapValue);\r\n      }\r\n      return true;\r\n    } catch (ParseException e) {\r\n      LOG.warn(\"Could not json-decode string: \" + line, e);\r\n      return false;\r\n    } catch (NumberFormatException e) {\r\n      LOG.warn(\"Could not parse field into number: \" + line, e);\r\n      return false;\r\n    }\r\n```\r\n\r\nA quick fix is to simply catch for a NPE in LzoJsonRecordReader#decodeLineToJson() and JsonLoader#parseStringToTuple() as is already being done for e.g. ParseException.  This fix has worked fine for us.  As I said above, however, this might not fix the root of the problem.\r\n\r\nI am not so familiar with the code, so I am not sure whether just catching NPEs is the best fix.  Maybe you have a better idea!","html_url":"https://github.com/kevinweil/elephant-bird/issues/65","title":"LzoJsonLoader and JsonLoader may fail with NullPointerException","number":65,"id":1295282,"created_at":"2011-07-27T12:01:20Z"},"action":"created","comment":{"user":{"avatar_url":"https://secure.gravatar.com/avatar/11c1a8701fcf16f2c8f8a6b7f519825c?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png","gravatar_id":"11c1a8701fcf16f2c8f8a6b7f519825c","url":"https://api.github.com/users/miguno","login":"miguno","id":294849},"updated_at":"2011-11-10T11:27:45Z","url":"https://api.github.com/repos/kevinweil/elephant-bird/issues/comments/2694166","body":"I sent you a pull request at https://github.com/kevinweil/elephant-bird/pull/98.\r\n\r\nThe commit is based on the eb-dev as of commit 06608de8d0.","id":2694166,"created_at":"2011-11-10T11:27:45Z"}}