Servlets :: Large File Upload From Client To Content Management System
Sep 3, 2009
I have a requirement where a large file (100 -200MB) is uploaded from the client to a content management system. I am using a servlet with Apache Commons File Upload API. Apache FileUpload has 2 ways of handling files,
1) Non-Streaming
2) Streaming
Currently I use the Non-Streaming approach where the servlet stores the file in a temp location and upload the same into the content management system - This is taking lot of time so I am trying to implement Streaming API.
Content Management API supports streaming in 2 methods,
a) SetContent - Takes the file's ByteArrayOutputStream as input -> This gives OutOfMemoryException because the file being large
b) AppendContent - Takes the file's ByteArrayOutputStream as input -> This method can be called multiple times to upload the large file but I dont know how to do this. The Apache File Upload gives InputStream of the file and I need to split that into chuncks and append into the content management system.
How to convert InputStream to 4KB ByteArrayOutputStream so that I can use the AppendContent method in content management API?
View Replies
ADVERTISEMENT
Nov 25, 2013
Our client has a user facing web application running on Jboss. There is a separate admin application (in its own ear) but deployed on same Jboss server on which user facing web application is running.
They need a screen to upload large amount of data into database. Their original files were in excel with size > 60 mb. We suggested following to them:
a. Change upload format to CSV - this brought down file sizes to 25-30 mb
b. Upload process will be MDB - asynchronous processing of data so that admin web app does not stop responding
We also suggested following to them:
a. Host admin app on a different machine so that user facing site does not respond slow during data processing
b. We can provide incremental upload feature and they should upload files in the chunks of 4-5 mb, specifically if they have user a web page to upload such files - they don't buy this argument though.
c. Data processing can be a separate script instead of a part of admin web application. They can FTP files to a designated location and this script will process those files.
I have following questions:
Q1 - Have you seen upload of such large datafiles to a web application? I see sites like Zoho CRM or Salesforce do not support such data imports and mostly fail or not respond.
Q2 - Is there a set of guidelines/best practices to upload large data files of this nature? How do insurance companies or others with enormous set of data accomplish such tasks (what is the architecture of such programs)?
View Replies
View Related
Dec 1, 2014
In my web application i want to upload file to drop-box. I am getting file name from browser.Is it possible to upload file to drop-box with only file name.
Below the drop-box upload code with java.
File inputFile = new File("New Text Document.txt");
System.out.println("inputFile.getAbsoluteFile(): " + inputFile);
FileInputStream inputStream = new FileInputStream(inputFile);
try {
DbxEntry.File uploadedFile = client.uploadFile("/magnum-opus.txt",
DbxWriteMode.add(), inputFile.length(), inputStream);
System.out.println("Uploaded: " + uploadedFile.toString());
} finally {
inputStream.close();
}
In the above code the place New Text Document.txt we have to provide total path of file.
View Replies
View Related
Oct 27, 2014
I'm in the middle of building A time management system for myself.I'm currently using JodaTime for timing in java. I just can't figure out who to save my data like, the starttime, stoptime and day time.
public void save() throws SQLException {
String checkSql = "select count(*) as count from Time where id=?";
PreparedStatement checkStmt = con.prepareStatement(checkSql);
String insertSql = "insert into Time (id, day, starttime, stoptime, overtime, exception, hours) values (?, ?, ?, ?, ?, ?, ?)";
PreparedStatement insertStatement = con.prepareStatement(insertSql);
String updateSql = "update Time set day=?, starttime=?, stoptime=?, overtime=?, exception=?, hours=? where id=?";
PreparedStatement updateStatement = con.prepareStatement(updateSql);
[code]....
View Replies
View Related
Dec 4, 2014
How to upload file to google drive using java. my java code is.
HttpTransport httpTransport = new NetHttpTransport();
JsonFactory jsonFactory = new JacksonFactory();
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(
httpTransport, jsonFactory, CLIENT_ID, CLIENT_SECRET, Arrays.asList(DriveScopes.DRIVE))
.setAccessType("online")
.setApprovalPrompt("auto").build();
[Code] ....
So in that file object(java.io.File fileContent = new java.io.File("document.txt");) asking complete file path. But in file upload we can get only file name not path.
View Replies
View Related
Jun 30, 2014
I have a code that uploads files in server after browsing folders and files then get the paths of files but I have a problem in getting the paths
List items = upload.parseRequest(request);
Iterator iterator = items.iterator();
while (iterator.hasNext()) {
FileItem item = (FileItem) iterator.next();
if (!item.isFormField())
{ fileName = item.getName();
root = getServletContext().getRealPath("/");
path = new File(root + "/uploads");
[Code]...
list1 must has paths that I want but I do not get the paths of upload files
View Replies
View Related
Sep 30, 2014
Hospital management system in java. I have designed Hospital, Doctor, Patient, Appointment classes. I am able to add doctor list and patient list and show the doctor list and patient list. How to take new appointment from a patient and assign it to a particular doctor and how to generate a report for inpatient and outpatient on a daily basis.
View Replies
View Related
Apr 7, 2015
I need to built a file upload service which should be memory effective. I should avoid loading the entire file into memory,Since I may have multiple http request which will pile up the Heap memory. Any effective way to upload a large file(For ex:1GB file) using http Streaming. I need to do the file upload on a single http call. Let's consider a scenario where 1 GB file to be uploaded using 512MB Heap memory. Not Sure If practically I can achieve this or not.
View Replies
View Related
Mar 14, 2015
I have the file in my project sitemap.xml, which i am trying to write via XMLStreamWriter. My code gets successfully executed as i can see the logs. But my sitemap.xml file keeps blank. Why nothing is getting write in my sitemap.xml file. Below is my servlet code.
@Override
protected void doGet(SlingHttpServletRequest request, SlingHttpServletResponse response)
throws ServletException, IOException {
logger.info("request.getResponseContentType() ::: " + request.getResponseContentType());
logger.info("root path is :: " + rootPath);
if (request.getResponseContentType() == null) {
[Code] ....
I can see "END Document" and "Location is " under my log file.
View Replies
View Related
Jan 10, 2015
I need to make some operations with a file. I struggle to understand why Apache is throwing this error:
(The system cannot find the file specified)
My file is located in the root directory of the project.
And here is how I indicate the path to it:
String filePath = "default.jpg";
InputStream inputStream = new FileInputStream(new File(filePath));
If I put the file in the Eclipse folder, it works... But I need it to work if I put it in my project's root folder.
View Replies
View Related
Jan 12, 2015
This issue is regarding response from the servlet
I have written a code to download .csv file with records from DB.
To download records i am uploading a .CSV file containing telephone number.
After downloading the .CSV file page is not getting refreshed.
Below is the code snippet i am using,where i am setting response content type as test/csv.
ServletOutputStream op = resp.getOutputStream();
// Set content type of output
resp.setContentType("text/csv");
resp.setHeader("Content-Disposition", "attachment; filename="test"");
op.flush();
op.close();
How that page will get refreshed after csv file download or after response.
View Replies
View Related
Apr 9, 2015
I would like to understand how does multipart/form-data works during file upload scenario's, Does it chunks the data from client to server while transferring the files ?
View Replies
View Related
Jan 16, 2014
I develop a finite element code at java. I am looking for efficiency solver and fast for large, sparse , symmetric and positive define matrices .
i used in jblas but i encounter in problem when the matrix has high condition number(ill condition number) and i get error for singular matrix while in mathematica i succeed to solve that system without problems...
Any good solver and fast solver package in java can i use for solving that system?
View Replies
View Related
Jan 25, 2014
Iam trying to upload jpeg image.
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
[Code] ....
In servlet the below code
DataInputStream din = new DataInputStream(request.getInputStream());
byte[] data = new byte[0];
byte[] buffer = new byte[50];
int bytesRead;
while ((bytesRead = din.read(buffer)) > 0 ) {
byte[] nData = new byte[data.length + bytesRead];
System.arraycopy(data, 0, nData, 0, data.length);
System.arraycopy(buffer, 0, nData, data.length, bytesRead);
data = newData;
}
What i see is servletinput stream does't have any values.It does't even enter while loop.
View Replies
View Related
Oct 8, 2014
We have a website used for downloading large files as large as 6gb and sometimes larger. We have all files in FTP server. The websit is JSP/Servlet /Tomcat combination. Below is code sample. I need a solution to increase the download speed. I understand it is bound to network bandwidth but are the steps that we need to take while we have such large files.I read about multipart downloads,gunzip streaming.
//first connect to FTP server and login and keep connection
Calendar cal=Calendar.getInstance();
startTime=cal.getTime();
fis=ftp.downloadFile(ftpclient,separator+resourceID);
response.setContentType("application/*");
response.setHeader("Content-Disposition", "attachment; filename="" + downloadFileName + "";");
byte[] bytes = new byte[1024];
[code]....
View Replies
View Related
Nov 14, 2014
I upload a file and save it like the following
byte[] uploadedFileBuf = new byte[(int) uploadedFile.length()];
saveFileSystem(excelUpload + getLocalFileName(acad_year, stuid), uploadedFileBuf);
private boolean saveFileSystem(String filename, byte[] data){
try{
FileOutputStream fos = new FileOutputStream(filename);
fos.write(data);
fos.close();
[Code]...
I can not read it use
HSSFWorkbook WorkBook = new HSSFWorkbook(new FileInputStream(filename));
View Replies
View Related
Jan 21, 2015
I need to upload a file in R, I can do it with a CSV file or excel ect. The file I received is already in R format .Rdata.It is a file called surveydata.rdata.The two data frames are surveydata.4.num.frame and surveydata.4.lab.frame.The surveydata.4.num.frame is what I need.I can click on it in Rstudio to see the data but I can't load it to run an analysis.How can I upload this?
View Replies
View Related
Mar 22, 2014
I'm having some problems styling the PrimeFaces upload tag <p:fileUpload /> in my application.
In my xhtml code I have the following:
<p:fileUpload id="fileUpload" fileUploadListener="#{filters.upload}"
allowTypes="#{filters.uploadTypes}" invalidFileMessage="#{filters.uploadBadType}"
sizeLimit="#{filters.uploadSize}" invalidSizeMessag="#{filters.uploadBadSize}"
update="fileUpload fileTable filterTab uploadMessage hiddenNum hiddenPhoto uploadError footer
namePhotSystem checkOverwrite"
description="Select Text File" disabled="#{filters.fileuploadDisabled}" fileLimit="1"
fileLimitMessage="You are only allowed to upload one file at a time"
styleClass="fileUploadClass" label="Choose the File to Upload" auto="true" />
This all works correctly, both the AJAX update and on the server, but I'm unable to get the button or buttons displayed in the way I need.
By including the following in my CSS:
.ui-fileupload.fileUploadClass {width:428px;}
.ui-icon-plusthick {display:none;}
.ui-button-text {width:200px; height:20px;}
the first sets the width of the button bar, and by looking at the HTML code generated by PrimeFaces, I discovered that the second hides the "+" to the left of the "Choose" button, which here has been replaced by the text "Choose the File to Upload". However, this text is displayed on two lines, with the word "Upload" on the second. I want the whole text to be on one line, but no matter what I do, I cannot get this to work properly. The third line of my CSS code does indeed reduce the height of the button, but the width does not work at all, and the last part of the text is simply lost. How do I put all the text on one line and adjust the width of the button?
Another question is that when I remove the auto="true" attribute from the tag, which I had originally, the "Upload" button is displayed to the right of the "Choose" button, and when a file is chosen but not yet uploaded, it is displayed below the button. This is a capability that I might still want, however, I am unable to style the display, and in particular the progress window to the right of the file name and size. All the files I want to upload are relatively small text files, and I want to adjust this window or even hide it completely. How is this done?
Incidentally, because the files are all quite small, I have disabled the display of the "Cancel" button, so either only the "Choose" button is shown, or the "Choose" and "upload" buttons.
View Replies
View Related
Apr 16, 2014
I am trying to implement a file upload functionality. One of the requirement is to display a 'Please Wait!!!' message to the user while the upload is going on. My project currently uses JSF 1.2, Tomahawk, JQuery 1.9, JQuery UI 1.10. I am aware that this can be accomplished using a4j with richfaces, however I am not supposed to add any new library to the existing project. how this can be done?
View Replies
View Related
May 20, 2014
I'm using jsf 2 to upload file, first I upload the file in a system directory, then trying to store the path to database with other information, my stuck is that when submitting I upload the file successfully, find it in the right place, find the other information such as description, file name ... in database but don't find the path. this is my managed bean :
package mbeans;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URL;
import java.util.List;
[code]...
View Replies
View Related
Apr 1, 2014
I have a JSF page (called MainPage) with a commandButton: clicking on it, I open a modal panel with a <Rich:fileUpload> component, related to a listener in corresponding bean.
When I start to upload a file, page MainPage starts to refresh, but I would like to prevent this because it's not necessary.
I tried to "play" with <Rich:fileUpload> property values, but nothing seems to work and I don't know what to do anymore.
Here is file upload component
<rich:fileUpload fileUploadListener="#{pannelloUploadBean.uploadListener}"
id="#{cid}_input"
ajaxSingle="true"
immediateUpload="true"
listHeight="200px"
listWidth="458px">
[Code] .....
View Replies
View Related
Apr 16, 2015
I am not all secure on the execution of the fileupload Primefaces.
The uploading execution occurs normally, but I would like to have a counter that control's the received files. The problem is that sending files go by so fast by this method that appears to be running in parallel. My question is whether this behavior is normal while sending files.
If it is not, where may be the error. If it is normal what can i do to make the counter really take the right value?
I'm using the scope of the "bean" type of view, but when im trying to run the session the problem also occurs depending on the file size.
If my files are very small, the execution is very fast and I cant follow the counter. If I clean the code the execution usually occurs no matter the size of the files.
@ManagedBean (Name = "bean")
@ViewScoped
public class BeanUpload {
private int counter;
@PostConstruct
public void init () {
counter = 0;
[Code] .....
The result on the console for 3 files is:
COUNTER EX: 1
COUNTER EX: 1
COUNTER EX: 1
If I run the code in debug mode on the console this is the result:
COUNTER EX: 1
COUNTER EX: 2
COUNTER EX: 3
Versions: Mojarra 2.1. Primefaces 5.1.15
View Replies
View Related
Dec 14, 2014
we have deployed application on web sphere server and using servlets and jsp only.
View Replies
View Related
Oct 22, 2014
Creating a file upload servlet to accept a CSV and parse for insert into a database, however, whenever I click submit, it always seems to open a new tab/window. Below is the method I have that builds the upload form: (Using GWT 2.4)
private void buildUpload(){
LayoutContainer headerContainer = new LayoutContainer(new ColumnLayout());
headerContainer.setStyleAttribute("padding", "5px");
add(headerContainer);
NamedFrame hiddenFrame = new NamedFrame("uploadFrame");
final FormPanel form = new FormPanel(hiddenFrame);
[Code] .....
Is there something I'm missing? or something I've added that makes it open a new tab/window?
View Replies
View Related
Mar 15, 2014
1. creates a file.
2. ask user to write into that file
3. save the file
4. print content of file.
Is my current exercise, so far i have gotten the code to create a file. What should i use to ask user to write into that file and save?
package assignment7;
import java.io.*;
public class Exercise2
{
public static void main ( String [ ] args ) {
String filePath="newfile.txt";
File newFile = new File ( filePath ) ;
[Code] .....
View Replies
View Related
Mar 17, 2014
1. creates a file.
2. ask user to write into that file
3. save the file
4. print content of file.
is my current exercise.so far i have gotten the code to create a file, and ask the user to input their age.what should i use to save what the user writes into the file?
Java Code:
package assignment7;
import java.io.*;
import java.util.*;
public class Exercise2
{
public static void main ( String [ ] args ) throws IOException
{
Scanner scan = new Scanner(System.in);
[code]....
View Replies
View Related
