multithreading - Why multithread program in Java slow and yet doesn't use much CPU time? -

my java program uses java.util.concurrent.executor run multiple threads, each 1 starts runnable class, in class reads comma delimited text file on c: drive , loops through lines split , parse text floats, after data stored :

static vector static concurrentskiplistmap 

my pc win 7 64bit, intel core i7, has 6 * 2 cores , 24gb of ram, have noticed program run 2 minutes , finish 1700 files, cpu usage around 10% 15%, no matter how many threads assign using :

executor executor=executors.newfixedthreadpool(50); 

executors.newfixedthreadpool(500) won't have better cpu usage or shorter time finish tasks. there no network traffic, on local c: drive, there enough ram more threads use, have "outofmemoryerror" when increase threads 1000.

how come more threads doesn't translate more cpu usage , less time of processing, why ?

edit : hard drive ssd 200 gb.

edit : found problem was, each thread writes it's results log file shared threads, more times run app, larger log file, slower gets, , since it's shared, slows down process, after stopped writing log file, finishes tasks in 10 seconds !

the outofmemoryerror coming java's own limits on memory usage. try using of arguments here increase maximum memory.

for speed, adam bliss starts suggestion. if same file on , over, imagine having multiple threads try read @ same time result in lot of contention on locks on file. more threads mean more contention, result in worse overall performance. avoid , load file once if it's possible. if it's large file, have 24 gb of ram. can hold quite large file, may need increase jvm's allowed memory allow whole file loaded.

if there multiple files being used, consider fact: your disk can read 1 file @ time. having multiple threads trying use disk @ same time won't effective if threads aren't spending time processing. since have little cpu usage, thread loads part of file, runs on part got buffered, , spends lot of time waiting rest of file load. if you're loading file on , over, still apply.

in short: disk io culprit. need work reduce threads aren't contending file content much.

edit:

after further consideration, it's more synchronization issue. threads getting held trying add result list. if access frequent, result in huge amounts of contention locks on object. consider doing having each thread save it's results in local list (like arraylist, not thread safe), , copying values final, shared list in chunks try reduce contention.