Reading Large Json Files to Objects in Java Using Gson

In the last couple of JSON tutorials for Java programmers, we have learned how to parse JSON using JSON-Uncomplicated library, parsing JSON array to Coffee array using GSon, and in this tutorial, we will learn how to parse a big JSON file in Java using Jackson'southward Streaming API. Jackson is one of the most pop JSON processing frameworks and provides three chief models to parse and process JSON information including Streaming API, data binding, and tree model. Out of these 3, Streaming works at the lowest level and can be used to parse huge JSON responses up to even gigabytes of size. If y'all are familiar with XML parsing, and then yous know that how difficult it is to parse huge XML files with DOM parser because it fully loads the file in memory earlier you can process it.

In instance you take low retention e.g. Android devices you can't utilize that to parse XML. Thankfully, XML provides SAX and StAX parsers which are streaming-based and can exist used to process huge files without loading them completely in memory.

Out of these two, StAX is fifty-fifty ameliorate because it allows pull-based processing where the customer pulls information from parser instead of parser pushing data, which is the case with SAX parser. Jackson'south Streaming API is similar to the StAX parser. Y'all can pull the information you want and ignore what yous don't want.

Though operation doesn't come without a cost, using Streaming API is a niggling difficult than using other Jackson models which provides a direct mapping betwixt Coffee and Jackson objects. You have to handle all JSON data by yourself while using Streaming API.

And, if you are new to JSON Parsing in Java and so I highly recommend you check out Jackson library. It'southward a very useful, versatile, and high-performance library for parsing JSON and CSV files and I think every Java programmer should know well-nigh it. If you demand a resource, check outJackson Quick Start: JSON Serialization With Java Made Piece of cake - a costless class on Udemy to learn Jackson API basics.

Benefits of using Jackson Streaming API

There are several advantages of using Jackson'south Streaming API to parse JSON String or convert Coffee object to JSON, but the most of import i is that its very efficient. It has to the lowest degree retention and processing overhead and is extremely useful to parse big JSON responses, for example a JSON response containing thousands of lodge or list of books or list of electronic items downloaded from e-commerce sites similar eBay or Amazon.

Talking nearly other two model of Jackson API, data binding model converts JSON to and from Java object based either annotation or Coffee bean convention, while Tree Model provides a mutable in-retentiveness tree representation of a JSON document, similar to DOM parser.

In brusk, Streaming API is virtually powerful, has less memory and CPU overhead but tricky to utilize, while data binding is frequently most convenient, on the other hand Tree Model is virtually flexible. BTW, both of this model internally uses streaming API to parse JSON strings before converting it into corresponding models.

Library JARs and Dependency

In order to effort post-obit example, you lot demand to download and add Jackson streaming API in your program's classpath. If you are using Maven and then you can add following dependency in your pom.xml file :

            <dependency>            <groupId>org.codehaus.jackson</groupId>            <artifactId>jackson-ninety</artifactId>            <version>1.9.12</version>            </dependency>          

or simply download and  add post-obit JAR in CLASSPATH of your Java application.

C:\.m2\repository\org\codehaus\jackson\jackson-xc\i.9.12\jackson-xc-ane.9.12.jar C:\.1000two\repository\org\codehaus\jackson\jackson-cadre-asl                           \ane.9.12\jackson-core-asl-1.9.12.jar C:\.mtwo\repository\org\codehaus\jackson\jackson-mapper-asl\1.9.12                        \jackson-mapper-asl-1.9.12.jar          

It's ofttimes easier to manage dependency using Maven and that'due south why I strongly suggest switching to Maven if you are not using it nonetheless. You can later upgrade to a newer version of Jackson library by just irresolute one line in Maven pom.xml file.

Parsing JSON in Java using Jackson Streaming API

This API has 2 primary module, 1 fore reading JSON and other for writing JSON and in this tutorial nosotros will learn both of them. JsonGenerator is used to write JSON while JsonParser is used to parse a JSON file. To demonstrate both reading and writing of JSON data in i program, I accept created two static methods, createJSON() and parseJSON(). Every bit proper name suggests first method creates a JSON file, which is then read by parseJSON() method. You can run into in the code that we are dealing with quite low level, we take not created whatsoever Java object to represent content of JSON, instead we are writing and reading String, numbers and arrays.

You can get an instance of JsonGenerator from JsonFactory class by calling createJsonGenerator() method. You tin besides provide the encoding you are intended to utilize, in our example I have used "UTF-8" which is a user-friendly default in most cases.

You lot tin use diverse write() methods to write contents.  Similarly, for parsing JSON, nosotros need to create an instance of JsonParser, which tin besides exist obtained from JsonFactory.

We parse JSON by calling nextToken() method of JsonParser in a while loop until nosotros reach JsonToken.END_OBJECT. Jackson API provides method to go name and value of token which you lot tin apply to identify data. Similarly while parsing JSON assortment, you lot await until you get JsonToken.END_ARRAY identifier.

Since we never load the whole file in memory, this method can exist used to read large JSON files with sizes from Megabytes to Gigabytes even with minimal retentiveness environment east.g. in Android smartphones or Java ME enabled devices.

Here is the sample code example to read and write JSON using Jackson Streaming API :

            import            java.io.File;            import            java.io.IOException;            import            org.codehaus.jackson.JsonEncoding;            import            org.codehaus.jackson.JsonFactory;            import            org.codehaus.jackson.JsonGenerationException;            import            org.codehaus.jackson.JsonGenerator;            import            org.codehaus.jackson.JsonParser;            import            org.codehaus.jackson.JsonToken;            import            org.codehaus.jackson.map.JsonMappingException;            /** * Java program to demonstrate how to use Jackson Streaming API to read and * write JSON Strings efficiently and fast. * *              @author              Javin Paul */            public            course            JsonJacksonStreamingAPIDemo{            public            static            void            main(String            args[]) {            System            .out.println("Creating JSON file past using Jackson                                  Streaming API in Java");         createJSON("jacksondemo.json");            Organization            .out.println("washed");            System            .out.println("Parsing JSON file by using Jackson                                  Streaming API");         parseJSON("jacksondemo.json");            System            .out.println("done");     }            /*      * This method create JSON Cord by using Jackson Streaming API.      */            public            static            void            createJSON(String            path) {            effort            {            JsonFactory            jsonfactory            =            new            JsonFactory();            File            jsonDoc            =            new            File(path);            JsonGenerator            generator            =            jsonfactory.createJsonGenerator(jsonDoc,            JsonEncoding                          .UTF8);              generator.writeStartObject();             generator.writeStringField("firstname",            "Garrison");             generator.writeStringField("lastname",            "Paul");             generator.writeNumberField("phone",            847332223);              generator.writeFieldName("address");              generator.writeStartArray();             generator.writeString("Unit of measurement - 232");             generator.writeString("Sofia Streat");             generator.writeString("Mumbai");             generator.writeEndArray();              generator.writeEndObject();              generator.close();            System            .out.println("JSON file created successfully");          }            catch            (JsonGenerationException            jge) {             jge.printStackTrace();         }            catch            (JsonMappingException            jme) {             jme.printStackTrace();         }            grab            (IOException            ioex) {             ioex.printStackTrace();         }     }            /*      * This method parse JSON String past using Jackson Streaming API case.      */            public            static            void            parseJSON(String            filename) {            try            {            JsonFactory            jsonfactory            =            new            JsonFactory();            File            source            =            new            File(filename);            JsonParser            parser            =            jsonfactory.createJsonParser(source);            // starting parsing of JSON String            while            (parser.nextToken()            !=            JsonToken                          .END_OBJECT) {            String            token            =            parser.getCurrentName();            if            ("firstname"            .equals(token)) {                     parser.nextToken();            //next token contains value            String            fname            =            parser.getText();            //getting text field            System            .out.println("firstname : "            +            fname);                  }            if            ("lastname"            .equals(token)) {                     parser.nextToken();            Cord            lname            =            parser.getText();            System            .out.println("lastname : "            +            lname);                  }            if            ("phone"            .equals(token)) {                     parser.nextToken();            int            phone            =            parser.getIntValue();                          // getting numeric field            Organisation            .out.println("phone : "            +            phone);                  }            if            ("address"            .equals(token)) {            Organization            .out.println("address :");                     parser.nextToken();                          // side by side token volition be '[' which ways JSON array            // parse tokens until y'all detect ']'            while            (parser.nextToken()            !=            JsonToken                          .END_ARRAY) {            System            .out.println(parser.getText());                     }                 }             }             parser.close();          }            catch            (JsonGenerationException            jge) {             jge.printStackTrace();         }            take hold of            (JsonMappingException            jme) {             jme.printStackTrace();         }            catch            (IOException            ioex) {             ioex.printStackTrace();         }     }          

and here is the output of our programme, when you run it from Eclipse or directly from the command line :

Creating JSON file by using Jackson Streaming API in Coffee JSON file created successfully washed Parsing JSON file by using Jackson Streaming API firstname : Garrison lastname : Paul telephone : 847332223 address : Unit of measurement - 232 Sofia Streat Bombay done

You will besides see file jacksondemo.json in your projection directory with the following JSON String :

{            "firstname":"Garrison",            "lastname":"Paul",            "telephone":847332223,            "address":["Unit - 232","Sofia Streat","Mumbai"] }

That'due south all about how to utilize Jackson Stream API to parse JSON String and to create JSON from Java objects. It's a powerful library with lots of features but Streaming is best. I know information technology's a little scrap difficult and y'all need to write a lot of code with hard-coded filed names, it is the fastest way to read a large JSON file in Java with less memory overhead. If you lot are dealing with normal size JSON output and you don't have retentiveness constraints then you can ever utilize Jackson Information binding model to parse JSON to Java Object.

Other JSON tutorials you may like to explore

• How to convert a JSON  Cord to POJO in Java? (tutorial)
• three Means to parse JSON Cord in Coffee? (tutorial)
• How to convert JSON array to String array in Java? (example)
• How to catechumen a Map to JSON in Java? (tutorial)
• How to utilise Google Protocol Buffer in Java? (tutorial)
• How to use Gson to convert JSON to Coffee Object? (example)
• 5 Books to Acquire Remainder and RESTful Spider web Services (books)

P.S. - If you are looking for online preparation to larn how to develop RESTful Web Services in Java using the Spring framework, I propose you joining Eugen Paraschiv's Residual with Spring course. The course has diverse options depending upon your experience level and how much you want to larn e.g. beginner'southward class, intermediate class, and main form. Y'all can join the one which suits you improve, though I suggest joining the master class if you are serious about becoming an skillful Java REST developer.

goldbergexch1973.blogspot.com

Source: https://javarevisited.blogspot.com/2015/03/parsing-large-json-files-using-jackson.html

Share this post