openmolecules.org

 
Home » DataWarrior » Cheminformatics » Out of memory
Out of memory [message #2395] Tue, 11 March 2025 16:10 Go to next message
Christophe is currently offline  Christophe
Messages: 34
Registered: January 2022
Member
Hello everyone,

I am currently working with a quite big dataset (~25000 molecules) and I try to make a UMAP vizualisation after the calculation of some data warrior fingerprints.
Each time I try to create a UMAP vizualization I get the following message:
"Out of memory. Launch DataWarrior with Java option -Xms???m or -Xmx???m"

How can I get around this problem?
I run DW on Intel core i5 at 2.7 GHz with 64 Go RAM

Thank you for any clarification you may be able to provide
Christophe
Re: Out of memory [message #2396 is a reply to message #2395] Wed, 12 March 2025 23:07 Go to previous messageGo to next message
nbehrnd is currently offline  nbehrnd
Messages: 233
Registered: June 2019
Senior Member
Hello Christophe,

the line you report is very close to the BASH script of DataWarrior's installer and launcher script in Linux. As an example, it can contain the line reading

/opt/datawarrior/jre/bin/java -Xms512m -Xmx4096m -splash:/opt/datawarrior/loading.png --add-exports java.base/jdk.internal.module=ALL-UNNAMED -Dfile.encoding=UTF-8 -cp $DWJAR/datawarriorlauncher.jar:$DWJAR/capka.jar org.openmolecules.datawarrior.launcher.DataWarriorLauncher $*

This example reads as "allocate DataWarrior 512 MByte RAM to start, provide up to 4 GO/GByte maximum" to allow other programs than DataWarrior to process concurrently. However, if you manage a larger data set on a more powerful computer, it is possible to manually adjust (i.e., increase) these thresholds. Try to work with e.g., `-Xmx60g`.

Best regards,

Norwid

https://www.star.bris.ac.uk/mbt/stilts/sun256/jvmArgs.html

[Updated on: Wed, 12 March 2025 23:08]

Report message to a moderator

Re: Out of memory [message #2399 is a reply to message #2396] Mon, 17 March 2025 20:52 Go to previous messageGo to next message
thomas is currently offline  thomas
Messages: 728
Registered: June 2014
Senior Member
25'000 Molecules is a rather small file. I am astonished. Do you have hundreds of columns? I just ran a UMAP from SkelSpheres (the largest of the vector descriptors) with 250'000 molecules, which also worked fine with just a few GB of -Xmx.

Which version and which OS do you use? On all platforms except Windows you can increase the -Xmx setting easily.
Re: Out of memory [message #2400 is a reply to message #2399] Wed, 19 March 2025 15:49 Go to previous messageGo to next message
Christophe is currently offline  Christophe
Messages: 34
Registered: January 2022
Member
Hello,

Thank you. Actually I'm running DW 6.04.02 on Windows 10. I can't find the file I have to edit to allocate more RAM on Windows.
Re: Out of memory [message #2404 is a reply to message #2400] Sat, 22 March 2025 17:08 Go to previous messageGo to next message
nbehrnd is currently offline  nbehrnd
Messages: 233
Registered: June 2019
Senior Member
Hello Thomas,

an answer on stackoverflow describes Windows' control settings to include a category "Java" to set the flags accordingly. Contrasting to my anticipation, on an instance of Windows 10 (64bit) with a working installation of DataWarrior (06.04.02), this however displays only sun's Java engine. Would one have to add `C:\Program Files\DataWarrior\jre\bin\java.exe` (or `javaw.exe`) to monitor (and eventually set the relevant flags) to then address DataWarrior's Java engine?

Best regards,

Norwid

https://stackoverflow.com/questions/70262178/defining-java-h eap-size-for-all-users-on-a-windows-system

[Updated on: Sat, 22 March 2025 17:09]

Report message to a moderator

Re: Out of memory [message #2405 is a reply to message #2404] Sat, 22 March 2025 17:26 Go to previous messageGo to next message
nbehrnd is currently offline  nbehrnd
Messages: 233
Registered: June 2019
Senior Member
Hello Christophe,

a bypass could be to set up a separate Linux partition on your computer because then the file in path /opt/datawarrior/datawarrior would provide you access to these parameters. Either by a double-boot (right after starting the computer, you will be asked which operating system to be used exclusively), or with a virtualization (for instance windows wsl) as a permanent change to the computer.

It equally is possible to start Linux from a USB/pen drive (e.g., https://rufus.ie/en/) provided the BIOS of the computer is set to allow to boot from USB. In principle this would provide a non-permanent access to the parameters (after launch of the OS, installation of DataWarrior, adjustment of the parameters, launch of DataWarrior) without change to the hosting computer. This tends to be a bit less performant than a true installation, though not as much with 64 Go at disposition, than e.g., 8 Go. After deconnection of the USB drive, the computer would boot as it did prior to this excursion. Which could be worth a test, too.

Best regards,

Norwid

https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux
Re: Out of memory [message #2407 is a reply to message #2405] Thu, 27 March 2025 15:29 Go to previous message
Christophe is currently offline  Christophe
Messages: 34
Registered: January 2022
Member
Thank you for your answers. I will try to run DW under Linux or find another solution.
Best regards,

Christophe
Previous Topic: comparing diversity
Goto Forum:
  


Current Time: Fri Mar 28 17:24:18 CET 2025

Total time taken to generate the page: 0.03248 seconds