After data has been collected, upon continuing on the phase of text parsing and database insertion, i get a memory overflow 8GB used !! and ubuntu hangs(can't resume without restart). Still investigating the issue....
After much investigation, the issue was based on how my script was developed. What happened was the script was trying to store all the data of file's accessed into an array before insertion into the database. The RAM couldn't handle this much information which caused a massive increase of memory utilization to 100%. So i resolved this issue by making sure that data is inserted per file accessed in the directory.
After much investigation, the issue was based on how my script was developed. What happened was the script was trying to store all the data of file's accessed into an array before insertion into the database. The RAM couldn't handle this much information which caused a massive increase of memory utilization to 100%. So i resolved this issue by making sure that data is inserted per file accessed in the directory.
No comments:
Post a Comment