How To Avoid OutOfMemory Exception In Memory Intensive Application In Java
Jan 16, 2014
We have developed a java application, whose primary objective is to read a file(input file), process it and convert it into set of output files. (I have given a generic description of our solution, to avoid irrelevant details).
This program works perfectly fine when the input file is of 4 GB, with memory settings of -Xms4096m -Xmx16384m in a 32 GB RAM
Now we need to run our application with the input file of size 130 GB.
We used a linux box with 250GB RAM and with memory setting of -Xms40g -Xmx200g (also tried couple of other variations) to run the application and hit OutOfMemory Exception.
At this stage of our project it's very hard to consider redesigning the code to accommodate hadoop ( or someother large scale data processing framework), also the current hardware configuration which we can afford is 250GB of RAM.
Any ways to avoid OutOfMemory Exceptions, what is the general practice when developing these kind of applications.?
The method I am trying to successfully write is startsWith(String s1, String s2)
The intention of the code is to return true if s2 starts with the string s1, and false otherwise. For example,
x.startsWith("Maplestory","Maple"); Should return true.
This code partially works, depending on the values entered into it. When it is supposed to return true, it does do this. When it is meant to return false, I get a StringIndex OutOfBounds Exception.
public class CC02{ String remove1(char c, String s){ String to_return=""; while(true){ if(s.equals("")) return to_return;
[Code] ....
The method I have written uses one other method from my code, which is "reverse". Any way to return false without getting this error.
I'm writing an IRC server in Java, however when a client tries to connect I get a ConcurrentModificationException trying to add them to an ArrayList. My code is Here: [URL]
I don't understand why I am getting a ConcurrentModificationException or what one is.
I am not a java developer, but I am using a java code that was available online to convert a large XML file to CSV file. The input file size is big, it is around 3GB. I got an error that it is out of memory, it is expectedly due to the large input file that i am trying to convert. Splitting of this file is not possible,
This is what I ran : xml2csv-conv data.xml data.csv
Error:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at com.sun.org.apache.xerces.internal.dom.DeferredDoc umentImpl.createChunk(Unknown Source) at com.sun.org.apache.xerces.internal.dom.DeferredDoc umentImpl.ensureCapacity(Unknown Source) at com.sun.org.apache.xerces.internal.dom.DeferredDoc umentImpl.createNode(Unknown Source) at com.sun.org.apache.xerces.internal.dom.DeferredDoc umentImpl.createDeferredTextNode(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.Abstrac tDOMParser.character
[code]....
Additional information: I am running this from a Windows8 64 bit machine with 8GB physical RAM.
I am integrating Struts hibernate tiles and spring configuration project. While running give me exception as
Apr 15, 2015 7:16:10 PM org.apache.catalina.core.AprLifecycleListener init INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:Program FilesJavajre1.8.0_40in;C:WindowsSunJavain;C:Windowssystem32;C:Windows;C:/Program Files/Java/jre1.8.0_40/bin/server;C:/Program Files/Java/jre1.8.0_40/bin;C:/Program Files/Java/jre1.8.0_40/lib/amd64;C:ProgramDataOracleJavajavapath;C:Program Files (x86)NVIDIA CorporationPhysXCommon;C:Program Files (x86)InteliCLS Client;C:Program FilesInteliCLS Client;C:Windowssystem32;C:Windows;C:WindowsSystem32Wbem;C:WindowsSystem32WindowsPowerShellv1.0;C:Program FilesMySQLMySQL Server
I have Stock Tracker application running, everything works just fine, but when i hit the List Users button, i get an SQL Exception which says "Column bot found". I tried searching for the exception from the internet, but the answers didn't suit my needs. below is the Stock Tracker DB i think the exception might be coming from.
public class StockTrackerDB { private Connection con = null;
// Constructor: makes database connection public StockTrackerDB() throws ClassNotFoundException,SQLException
"A common memory matching game played by young children is to start with a deck of cards that contain identical pairs. For example, given six cards in the deck, two might be labeled
1, two might be labeled 2 and two might be labeled 3. The cards are shuffled and placed face down on the table.
The player then selects two cards that are face down, turns them face up, and if they match they are left face up. If the two cards do not match they are returned to their original position face down. The game continues in this fashion until all cards are face up.Write a program that plays the memory matching game. Use sixteen cards that are laid out in a 4x4 square and are labeled with pairs of numbers from 1 to 8. Your program should allow the player to specify the cards that she would like to select through a coordinate system.For example in the following layout:
All of the cards are face down except for the pair 8 which has been located at coordinates (1,1) and (2,3). To hide the cards that have been temporarily placed face up, output a large number of newlines to force the old board off the screen.Use 2D array for the arrangement of cards and another 2D array that indicates if a card is face up or face down.Or, a more elegant solution is to create a single 2D array where each element is an object that stores both the cards value and face.Write a function that shuffles the cards in the array by repeatedly selecting two cards at random and swapping them.
Note:To generate a random number x, where 0<= x <1, use x=Math.random();.For example, multiplying y six and converting to an integer results in an integer that is from 0 to 5."I have been thinking about the algorithm and design of the question for a few hours.
convert or move standalone java thread application into Tomcat server container for accessing its JNDI services? Also is it possible to schedule this thread application in Tomcat server? is it possible to keep this app in tomcat as web application and schedule in window's scheduler.
I would like to know how much of memory is free and used my java program.I have used Runtime.totalMemory and freeMemory() functions. However, how come used memory is so different than java.exe process show in windows task manager?
i have to write more than 100000 rows in a excel sheet (file size more than 20 MB) via java.
when i use XSSF, i am getting below Error.
java.lang.OutOfMemoryError: Java heap space at org.apache.xmlbeans.impl.store.Saver$TextSaver.resize(Saver.java:1592) at org.apache.xmlbeans.impl.store.Saver$TextSaver.preEmit(Saver.java:1223) at org.apache.xmlbeans.impl.store.Saver$TextSaver.emit(Saver.java:1144)
[Code]....
when i use HSSF , i am getting the below Error. java.lang.OutOfMemoryError: Java heap space
I have tried increasing the java heap size , by giving upto -Xms1500m -Xmx2048m
I and a friend are working with a project to create a file system, who manages a secondary memory simulated as a byte array in Java. We want the file system to be a hierarchical tree structure like in UNIX.
We have come quite far, but the paths are not handled correct. I seem to have mistaken the relative folder ./ for the root folder, but it should mean "working directory folder", ie, where I stand now. That is, if I stand in /dir1 as my "working directory" and make mkdir ./dir2 then should dir2 end up as subfolder in dir1. But with me it appears in the root.
I need to process 10000 xml files and verify and insert the data into database. I am loading all the files in the file object and iterating one by one. I am getting the memory issue. How to handle this?
I have been playing around with my code, but how to avoid NullPointerexception.. So my program's point is simple, use Jsoup to scrape html of certain webpage, then i search the things i want and print them out. Problem is, when scraped html doesnt contain even 1 thing on my search list, i get NullPointerException... i understand why, Heres part of my code:
Java Code:
//Things i need to search from html String[] MySearchArray = new String[]{"138","146","474"}; //Search things contained in MySearchArray and print them out for (String Ml : MySearchArray) { Element flights = doc.select(String.format("tr:contains(%s)", Ml)).first(); Elements flights2 = flights.select("td"); System.out.println(flights2.get(4).text() + " " + flights2.get(0).text()+ " " + flights2.get(3).text()); } mh_sh_highlight_all('java');
I have a HashMap with multiple values at runtime. In that map one key has empty value , how to avoid this value to add in a list. some code sample is below:
public HashMap getLoop2Map(Map map, String xslFile , int sheetNo){ HashMap hashMap = new HashMap(); try{ ArrayList list = new ArrayList(); System.out.println("-------Map : " +map); list.add(map.values()); //System.out.println("------- boolean : " +val); System.out.println("------List : " +list); }
I do not want to add the empty value in a list ...
result in map
-------Map : {Free Text Entry={}, Mouth / Throat={Free Text Entry={Free Text Entry=<FORMFILENAME>EditChartPhysicalExamText.form </FORMFILENAME><TAG>"<MUSCULOSKELETAL.PE>"</TAG>}, Salivary Glands Condition={Salivary Glands Condition=MouthAttribute}, Examination Overview- Mouth / Throat={Examination Overview- Mouth / Throat=MouthAttribute}, Tonsils={Tonsils=MouthAttribute},
I'm struggling with that piece of code, my intention is to check for the object I want to add before adding it, so there won't be any duplicate on my list. I'm not sure how could I do that, since I'm working with objects.
Person is a class with few parameters such as id, name, and few others.
I guess I should search for a person with the same id, since that has be unique, but can't get it right.
private ArrayList<person> model= new ArrayList<>(); //... if (model.contains(person))throw new IllegalArgumentException("duplicate"); else model.addElement(person);
I wanted to make a small program to move a small rectangle by pressing the WASD keys. The program works, except that when I hold a key to move the rectangle, after a second, auto-repeat starts up, and the rectangle motion accelerates. I want to prevent automatic repeat to activate, so that the rectangle moves at a constant speed when I hold a key and stops when I released. Here is the ButtonMotion classe :
I have code which query value from database,the use case is the user can enter value as 1,01,11 in database but when the user enter value in xml file he can only enter 11,01,12 in database there is two columns lets say column test1=1 and test2=2 combination of this is 12,which is the value the use will enter in xml, but sometime the use can enter test1=04 than column test2=00 ....
I'm using EclipseLink, WildFly, EJB, postgresSQL and JSF.I'm trying to persist some countries and their localities.So I've:
- Two entities Countries and Localities in which I specify respective columns and relations. - Abstract Session beans for entity classes: AbstractFacade providing basic crud methods and entity manager. - Two concrete session bean for entity classes: CountriesFacade and LocalitiesFacade. - A JSF managed bean named geoJSF. - A JSF page with a form allowing to insert new country and localities.
In geoJSF I'm injecting via EJB CountriesFacade as property named cf and LocalitiesFacade as property named lf.For the actual inserting country and locality I'm using geoJSF.country and geoJSF.locality. When the form is submitted I'm simply doing:
this.cf.create(this.country); this.locality.setCountry(this.country); //<- this throw an exception (unique constraint violation) due to the attempt to reinsert this.country this.lf.create(this.locality);
I disabled all cascade among relations definitions.Based on what I know this.country should appear detached to entity manager so, setting relation the entity manager try to re-persist it.
I have a practice program (written with NetBeans IDE 7.4) that calls methods to:
(1) fill an array with user input values
(2) sort the array values into ascending order
(3) print the array in assorted order.
(4) print the array without any duplicates
Method (4) prints the contents of the array without printing any duplicates. That is, if a number in the array has already been printed, it will not be printed again.
Method (4) seems to work, but may be inefficient or I may have done too much work making it difficult on myself or making the logic too confusing.
Is there a better way to do this WITHOUT using built-in functions in Java's library?
(Such as writing this method using for loops and counter variables: As practice I am supposed to avoid using Deleting functions for duplicates)
Here is my code:
public static void Duplicates(int [] array){ int duplicates = 1; String Output = ""; for(int i = 0; i < array.length -1; i++) { if (array[i] != array[i+1]){ duplicates ++;
