Reading A Large JSON File In Java
Jan 5, 2014
reading a large json file which is really huge and it contains pair of id and URL in a array as shown below.. i need to get this all id and URL from this file in my java code and need to perform operations . how to read such a big file and store the contents in java code to process it further.
{
"Array": [
{
"id": "1test",
"URL": "http://servername/test/1test.html"
},
{
"id": "2test",
"URL": "http://servername/test/2test.html"
}
....................many id and URL in the same Array which makes the file Huge..
]
}
View Replies
ADVERTISEMENT
Aug 14, 2014
I want to read json file as follow;
{
"M": {
"row": [
{
"col1": "c00"
},
{
"col1": "c10",
"col2": "c11"
},
{
"col1": "c20",
"col2": "c21",
"col3": "c22"
}
]
}
}
Next to reading I need to assign to two dimensional array, but without giving "col1","col2","col3" a lot.
The array is for example, Array[3][] = {{"c00"},{"c10","c11"},{"c20","c21","c22"}};
View Replies
View Related
Aug 13, 2014
I am using java process to start a system command in windows
Runtime r = Runtime.getRuntime();
Process pr = r.exec(cmdString);
I want to get the prompt out put from cmdString = "cmd /c type fileSmallSize"->>>> It is ok the have the content of the file when file is small.
However, for a large file java process will hang and no Exception occurred, what is the problem?
The easiest testing you can try on the logging.properties file in java.
public static String executeCmdAndReturnPromptResult(String cmdString)
throws Exception {
LOGGER.entering(CLASSNAME,
"Entering executeCmdAndReturnPromptResult()", cmdString);
String cmd = cmdString;
[Code] ....
It seemed to me that the bufferSize is limited so that I can only have it less than a default one, how to increase it?
My question now is how to increase the size of buffer in order to read a larger InputStream ?
BufferedInputStream() default size is
private static int defaultCharBufferSize = 8192;
private static int defaultExpectedLineLength = 80;
How to make it larger and working? I tried to increase the defaultCharBufferSize to 500000000 but it did not work!
View Replies
View Related
Sep 22, 2014
Im trying to read a file called m1 and put that information into a 2-D array with the first two numbers being the size of the array i was able to set the size of the array but how do i populate array?
import java.util.Scanner;
import java.io.*;
public class test{
public static void main(String [] args)throws IOException{
Scanner fr = new Scanner(new File("m1.txt"));
String line = fr.nextLine();
[code]....
View Replies
View Related
May 5, 2015
I'm currently working on a program what is reading a txt file with a bunch of numbers in it but keep getting an error back.
Validator.java:
import java.io.IOException;
import java.io.File;
import java.util.Scanner;
public class Validator {
public int[] accounts = {};
[Code] ....
Accounts.txt:
5658845
8080152
1005231
4520125
4562555
6545231
7895122
5552012
3852085
8777541
5050552
7576651
8451277
7825877
7881200
1302850
1250255
4581002
View Replies
View Related
Aug 27, 2014
I would like some clarification on a project I am working on. I need to build a java application to query a xml and json file. I have started by parsing both documents and I am unsure if I am doing the next steps correct. I was thinking about dropping the txt file with both outcomes into a db and doing it that way, is this the right way or is there a better and more efficient way of doing things ?
View Replies
View Related
Oct 7, 2014
I am trying to make a bank program that reads to .txt files, one that has the right format and one that has the wrong format. This is part of my problem I do not know how to do this. I also do not know why my code works if I input "account" rather than "account.txt" which is supposed to be the right thing.
Let me present my code that I have so far:
// File: BankAccount.java
import java.util.Scanner;
import java.io.IOException;
public class BankAccount
{
private int accountNumber;
private double balance;
[Code] ....
View Replies
View Related
Jun 4, 2014
I want to read and write json to/from a file. I read the following document:
JsonReader (Java(TM) EE 7 Specification APIs)
I try to define a JsonReader object in Eclipse:
JsonReader jr
But there appears to be no import for it. Where I am supposed to get the Json library?
View Replies
View Related
May 17, 2014
I have written my program result into a json file but i am getting json file in below format.
{"one":"one","two":"two","three":"three"}
but I have lot of entries to write into json finally it become unreadle format which is not very compart to read.
Is there any way to format or writing line by line into json file like below format,
{
"one":"one",
"two":"two",
"three":"three"
}
View Replies
View Related
Apr 23, 2015
I am advised to use a while loop the Scanner method hasNextLine() and in the while loop body, call the Scanner method nextLine(), and add the returned String to the ArrayList of Strings. what I have gotten so far:
Scanner input = new Scanner(new File(""));
while(input.hasNextLine()) {
String line = input.nextLine();
System.out.println(line);
View Replies
View Related
Feb 8, 2014
We have an autosys job running in our production on daily basis. It calls a shell script which in turn calls a java servlet. This servlet reads these files and inserts the data into two different tables and then does some processing. Java version is 1.6 & application server is WAS7 and database is oracel-11g.
We get several issues with this process like it takes time, goes out of memory etc etc. Below are the details of the way we have coded this process.
1. When we read the file using BufferedReader, do we really get a lot of strings created in the memory as returned by readLine() method of BufferedReader? These files contain 4-5Lacs of line. All the records are separated by newline character. Is there a better way to read files in java to achieve efficiency? I couldnt find any provided the fact that all the record lines in the file are of variable length.
2. When we insert the data then we are doing a batch process with statement/prepared statement. We are making one batch containing all the records of the file. Does it really matter to break the batch size to have better performance?
3. If the tables has no indexes defined nor any other constraints and all the columns are VARCHAR type, then which operation will be faster:- inserting a new row or updating an existing row based upon some matching condition?
View Replies
View Related
Aug 21, 2014
I am trying to learn how to parse a Json with java. So I have this code
import java.net.*;
import java.io.*;
public class test2 {
public static void main(String[] args) throws Exception {
[Code] .....
and it has this output
{
"response": {
"version":"0.1",
"termsofService":"http://www.wunderground.com/weather/api/d/terms.html",
"features": {
"conditions": 1
[code]...
How do I turn this information into java objects?
View Replies
View Related
Sep 13, 2014
//compiler error I'm receiving
J:CS3Student Folder Review Lab #2APlusImage.java:41: error: cannot find symbol
xcor.add(read.nexInt());
[Code].....
View Replies
View Related
Nov 25, 2013
Our client has a user facing web application running on Jboss. There is a separate admin application (in its own ear) but deployed on same Jboss server on which user facing web application is running.
They need a screen to upload large amount of data into database. Their original files were in excel with size > 60 mb. We suggested following to them:
a. Change upload format to CSV - this brought down file sizes to 25-30 mb
b. Upload process will be MDB - asynchronous processing of data so that admin web app does not stop responding
We also suggested following to them:
a. Host admin app on a different machine so that user facing site does not respond slow during data processing
b. We can provide incremental upload feature and they should upload files in the chunks of 4-5 mb, specifically if they have user a web page to upload such files - they don't buy this argument though.
c. Data processing can be a separate script instead of a part of admin web application. They can FTP files to a designated location and this script will process those files.
I have following questions:
Q1 - Have you seen upload of such large datafiles to a web application? I see sites like Zoho CRM or Salesforce do not support such data imports and mostly fail or not respond.
Q2 - Is there a set of guidelines/best practices to upload large data files of this nature? How do insurance companies or others with enormous set of data accomplish such tasks (what is the architecture of such programs)?
View Replies
View Related
Sep 9, 2014
How can I convert JSON string to HashMap. My JSON string is like
{
"AvailableDeliveries": {
"500": {
"code": "INOFFICE",
"desc": "In Office",
"id": 500,
[code]....
I looked on other examples which have collection of object but I have this extra top level object "AvailableDeliveries" not sure how to handle that level.
View Replies
View Related
Apr 7, 2015
I need to built a file upload service which should be memory effective. I should avoid loading the entire file into memory,Since I may have multiple http request which will pile up the Heap memory. Any effective way to upload a large file(For ex:1GB file) using http Streaming. I need to do the file upload on a single http call. Let's consider a scenario where 1 GB file to be uploaded using 512MB Heap memory. Not Sure If practically I can achieve this or not.
View Replies
View Related
Sep 3, 2009
I have a requirement where a large file (100 -200MB) is uploaded from the client to a content management system. I am using a servlet with Apache Commons File Upload API. Apache FileUpload has 2 ways of handling files,
1) Non-Streaming
2) Streaming
Currently I use the Non-Streaming approach where the servlet stores the file in a temp location and upload the same into the content management system - This is taking lot of time so I am trying to implement Streaming API.
Content Management API supports streaming in 2 methods,
a) SetContent - Takes the file's ByteArrayOutputStream as input -> This gives OutOfMemoryException because the file being large
b) AppendContent - Takes the file's ByteArrayOutputStream as input -> This method can be called multiple times to upload the large file but I dont know how to do this. The Apache File Upload gives InputStream of the file and I need to split that into chuncks and append into the content management system.
How to convert InputStream to 4KB ByteArrayOutputStream so that I can use the AppendContent method in content management API?
View Replies
View Related
Feb 18, 2014
I have a large text file of 1 GB size. I need to print the line when a matching word is found in a particular line. Below is the code I am using. But if there are many lines that has the matching word, it's taking lot of time. Any solution to print the lines much faster.
Scanner scanner = new Scanner(file);
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
if(line.contains("xyz")) {
System.out.println(line);
}
}
View Replies
View Related
May 21, 2013
I need to download large amount files, what should I care, Is the following code have problem on download large files
private static byte[] load(String filename) throws IOException {
FileInputStream fis = new FileInputStream(filename);
byte[] data = new byte[fis.available()];
fis.read(data);
fis.close();
return data;
}
View Replies
View Related
Dec 9, 2014
I am working on a project that requires me to build a database with random access file, representing products, the base product contains a name (about 30 characters), a price (double), and a quantity (integer). I have worked on this project for probably 15+ hours and have tried so many things and feel like I've barley made any progress...
The part i am really struggling with is taking the data from the text file and creating an object array with it using the product class. Once ive accomplished that, i have to use that data to create a random access file with the data. Here is the base Product class that must be used to create the objects for the array.
public class Product
{
public String pName;
public String stringName;
public double price;
public int quanity;
[Code]...
these continue for about 40-50 entries, they are not seperated by a blank line though i had to add those so it would display correctly, each entry is on its own line with name seperated with spaces, then price after a comma, then quanity after the second comma.....
View Replies
View Related
Dec 8, 2014
I am working on a project that requires me to build a database with random access file, representing products, the base product contains a name (about 30 characters), a price (double), and a quantity (integer). I have worked on this project for probably 15+ hours and have tried so many things and feel like I've barley made any progress...
The part i am really struggling with is taking the data from the text file and creating an object array with it using the product class. Once ive accomplished that, i have to use that data to create a random access file with the data.
Here is the base Product class that must be used to create the objects for the array.
public class Product
{
public String pName;
public String stringName;
public double price;
public int quanity;
//Constructor
public Product( String pName, double price, int quanity )
[code]....
and then here is the data from the text file that i must extract to use to create product objects.
Dill Seed,938,34
Mustard Seed,100,64
Coriander Powder,924,18
Turmeric,836,80
Cinnamon (Ground Korintje),951,10
Cinnamon (Ground) Xtra Hi Oil (2x),614,31
Cinnamon (Ground) High Oil (1X),682,19
these continue for about 40-50 entries, they are not separated by a blank line though i had to add those so it would display correctly, each entry is on its own line with name separated with spaces, then price after a comma, then quanity after the second comma.....
View Replies
View Related
Mar 6, 2014
i have to write more than 100000 rows in a excel sheet (file size more than 20 MB) via java.
when i use XSSF, i am getting below Error.
java.lang.OutOfMemoryError: Java heap space
at org.apache.xmlbeans.impl.store.Saver$TextSaver.resize(Saver.java:1592)
at org.apache.xmlbeans.impl.store.Saver$TextSaver.preEmit(Saver.java:1223)
at org.apache.xmlbeans.impl.store.Saver$TextSaver.emit(Saver.java:1144)
[Code]....
when i use HSSF , i am getting the below Error.
java.lang.OutOfMemoryError: Java heap space
I have tried increasing the java heap size , by giving upto -Xms1500m -Xmx2048m
View Replies
View Related
Jan 16, 2014
I develop a finite element code at java. I am looking for efficiency solver and fast for large, sparse , symmetric and positive define matrices .
i used in jblas but i encounter in problem when the matrix has high condition number(ill condition number) and i get error for singular matrix while in mathematica i succeed to solve that system without problems...
Any good solver and fast solver package in java can i use for solving that system?
View Replies
View Related
Jan 18, 2014
I have a CSV file with 16K entries of a data table. Does Java work well with CSV file? So I found this code. And it seems its quite easy to read in the data I need. Say for example if I wanted a loop to randomly pick the first field of a specific line in the CSV data table. How would i go about coding that??????
package cvs.test;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
[code]....
The CSV looks like the above. and I basically would like to read in the Hand to get it to show in a text box and then randomly have the program ask me to correctly identify the True/False return for one of the SB/BB/UG/MP/CO/BN columns.
View Replies
View Related
Mar 3, 2013
how to get the first few hex symbols of a file in java, for example if i input a pdf into my coding i want my program to output, e.g "25 46 44 38" ....
I have been able to print out the hex of a whole file but not managed to set a maximum read limit so that my code only takes a certain amount of values ....
View Replies
View Related
Mar 21, 2014
I am implementing a recursive descent parser that recognizes strings in the language below. The input should be from a file "input.txt" and output should be to the console.
The grammar:
A -> I = E | E
E -> T + E | T - E | T
T -> F * T | F / T | F
F -> P ^ F | P
P -> I | L | UI | UL | (A)
U -> + | - | !
I -> C | CI
C -> a | b | ... | y | z
L -> D | DL
D -> 0 | 1 | ... | 8 | 9
An example session might look like this:
String read from file: a=a+b-c*d
The string "a=a+b-c*d" is in the language.
String read from file: a=a**b++c
The string "a=a**b++c" is not in the language.
Java Code: /**
* The Grammar
* A -> I = E | E
*E -> T + E | T - E | T
*T -> F * T | F / T | F
*F -> P ^ F | P
*P -> I | L | UI | UL | (A)
*U -> + | - | !
[code]....
My current output looks like this:
Java Code: The string read from file: a=a+b-c*d
The string "" is not in the language. mh_sh_highlight_all('java');
So it seems to be reading the input file correctly. My error seems to be on this part
Java Code: s = args.length == 1 ? args[0] : ""; mh_sh_highlight_all('java');
When I put the string in that line such as: s = args.length == 1 ? args[0] : "a=a+b-c*d";
My output is:
Java Code: The string read from file: a=a+b-c*d
The string "a=a+b-c*d" is in the language. mh_sh_highlight_all('java');
How can I go about it reading the string from the input file though?
Also, I am not too sure my boolean functions for P, I and, L are correct. Mainly when two terms are together (UI, UL, CI, DL).
View Replies
View Related