ckegan
Hi (new to the forum and TOPAS beginner)
I have very large data sets (several hundred thousand diffraction patterns) that I need to process. I have successfully managed to set-up batch processing using tc.exe via a matlab script (as opposed to windows batch file), but I have a couple of questions on how to optimise the processing:
1) Is it possible to do parallel processing by calling several instances of tc.exe simultaneously?
2) Is it possible for TOPAS to read data in from memory rather than a file?
Any help appreciated. Thanks
Chris
alancoelho
For parallel processing it depends I think on the licenses. If you are using a Bruker dongle then it maybe possible to run more than one instance of the program. Try it.
Reading data from memory is not possible. You could use a RAM drive which uses memory but I'm not sure you will gain that much more speed. The point is that even though the data may be in memory the program needs to think that its reading a file.
The actual loading of TC.EXE into memory is probably more important that the data in memory. Whether TC.EXE stays in memory from one run to another is computer/operating system dependent. On my computer TC.EXE seems to stay in memory as running a dummy batch file with 1000 TC.EXE runs with iters 0 executes in 29 seconds, 0.029 seconds per run. This time is probably spent on start-up code such as the loading of RTTEM9.DXX etc...
codymconnors
Batch processing has been associated with mainframe computers since the earliest days of electronic computing in the 1950 s. There were a variety of reasons why batch processing dominated early computing. One reason is that the most urgent business problems for reasons of profitability and competitiveness were primarily accounting problems, such as billing. Billing may conveniently be performed as a batch-oriented business process, and practically every business must bill, reliably and on-time. Also, every computing resource was expensive, so sequential submission of batch jobs on punched cards matched the resource constraints and technology evolution at the time. Later, interactive sessions with either text-based computer terminal interfaces or graphical user interfaces became more common. However, computers initially were not even capable of having multiple programs loaded into the main memory.