Šī¸ 29.09.2020 Thomas Dreibholz, University of Duisburg-Essen / Institute for Experimental Mathematics / Computer Networking Technology Group
University of Duisburg-Essen /
Institute for Experimental Mathematics /
Computer Networking Technology Group
Homepage of Thomas Dreibholz /
Thomas Dreibholz's SimProcTC - A Powerful Tool-Chain for Setup, Distributed Processing and Analysis of OMNeT++ Simulations
Version 1.1.0 is available for download!
In order to efficiently perform simulations using my RSerPool simulation model RSPSIM, I have also developed a model-independent, flexible and powerful tool-chain for the setup, parallel run execution, results aggregation, data analysis and debugging - completely based on Open Source software. Due to its independence of a specific model, it may also be useful for many more users of OMNeT++. Therefore, I have released this tool-chain as Open Source under GPLv3 license.
The following items are a step-by-step installation guide for the simulation tool-chain.
Get the latest version of OMNeT++ here and install it under Linux. If you do not have Linux installed already, you may find my Little Ubuntu Linux Installation Guide helpful. This installation guide also provides help on how to install OMNeT++ on an Ubuntu system.
Note that while OMNeT++ also works under Microsoft Windows, my tool-chain has not been tested under this operating system yet. In particular, run distribution using RSerPool will not work under Windows unless you port the RSPLIB RSerPool implementation to Windows.
After installing OMNeT++, make sure that it is working properly.
Install GNU R. Usually, it will be available for your Linux distribution as installation package. However, if you decide to install it from source, you can download the source here.
Under Ubuntu/Debian Linux, you can download and install GNU R using the following command line:
sudo apt-get install r-base r-base-html r-base-latex r-base-dev r-doc-info r-doc-pdf r-doc-html r-mathlib
After installation, you can start GNU R by:
R --vanilla
You can quit GNU R using Ctrl+D.
The simulation tool-chain requires LIBBZ2 for compression and decompression of files. In particular, also the developer files (include files) of this library are required to compile the tool-chain. Usually, it will be available for your Linux distribution as installation package. However, if you decide to install it from source, you can download the source here.
Under Ubuntu/Debian Linux, you can download and install LIBBZ2 using the following command line:
sudo apt-get install libbz2-dev
Get the simulation tool-chain package from the Download section and unpack it. Also read the description paper in the docs/ directory; it gives important information on what the tool-chain actually does! The tool-chain archive includes the files of the tool chain as well as a small example simulation. The files have the following purposes:
In order to compile tool-chain and examples, call the following commands in the corresponding demo subdirectory (i.e. "demo4" for OMNeT++ 4.1 or "demo" for OMNeT++ 3.x):
opp_makemake -f -u Tkenv make cd tools make cd ..
Make sure that everything compiles successfully. Otherwise, the tool-chain will not work properly! After compilation, you can start the demo simulation by calling:
./demo4
The demo simulation packaged with SimProcTC simply presents the effects of fragmenting large packets into cells and forwarding them: the delays will significantly reduce at the price of increased overhead. Have a look into scenario.ned to see the parameters of the model:
An example simulation for this model is defined in test1.R: for each parameter of the model, the list "simulationConfigurations" contains a list with the parameter name as first element and its value(s) as further elements. For example, list("sourcePayloadSize", 1000, 2500) means that the parameter "sourcePayloadSize" should be used with the values 1000 and 2500. For each parameter combination, a separate run will be created. Furthermore, the variable "simulationRuns" specifies how many different seeds should be used. That is, for simulationRuns=3, runs for each parameter combinations are created with 3 different seeds (i.e. tripling the number of runs!).
The actual output of .ini files is realized in simulate-version1.R. Have a look over this file first, it should be quite self-explaining! In the function demoWriteParameterSection(), the actual lines for the parameters above are written for each simulation run. "simCreatorAdditionalActiveVariables" defines for which variables a table row should always be written. For example, if you always use cellHeaderSize=4, the createsummary script would neglect this parameter in the output table. Since it may be useful for your post-processing, you can add it to "simCreatorAdditionalActiveVariables". Note, that "simCreatorWriteParameterSection" is set to "demoWriteParameterSection". In the generic simulation.R script, always the names simCreatorXXX instead of demoXXX are used. In order to be model-independent, it is necessary to set these variables to the actual model-dependent functions in simulate-version1.R! When you adapt the tool-chain to you own model, you only have to create your own simulation-versionX.R script and leave the other scripts unmodified.
The variables "distributionPool" and "distributionProcs" in test1.R are used to control the request distribution. They will be explained later. For now, make sure that distributionProcs is set to 0! This setting means that all runs are processed on the local machine. Furthermore, for batch execution, you have to compile you model using the Cmdenv environment first:
opp_makemake -f -u Cmdenv make
Now, in order to perform the simulation defined in test1.R, simply execute test1.R using R:
R --vanilla < test1.R
The script will now create an .ini file for each run and a Makefile containing all runs. Finally, "make" will be called to process the created Makefile. "make" will already be called with the -j parameter corresponding to your number of CPUs/cores, so that it fully utilizes the computation power of your machine. You can observe the progress of the simulation processing by monitoring the log file:
tail -f test1/make.log
You can abort the simulation processing and continue later. Only the run(s) currently in progress are lost and have to be re-processed upon resumption. Already completed runs are saved and no re-processing is necessary.
After processing the simulation defined by test1.R, you can plot the results using plot-test1.R:
R --vanilla < plot-test1.R
The results will be written to test1.pdf (the file name will be the simulation output directory + .pdf). You can view it with any PDF reader, e.g. kpdf. The plotter settings at the head of plot-test1.R should be almost self-explaining. For "colorMode", you can also use cmBlackAndWhite or cmGreyScale. Setting "plotOwnOutput" to TRUE results in an own output file for each plot (instead of a single PDF file). "plotConfigurations" contains the definitions for each plot, in particular title, output file name for "plotOwnOutput", x- and y-axis ticks, legend position and the results data for each axis given by a template. A set of model-specific templates is already defined in simulate-version1.R, you can add additional ones there or to "plotVariables" in plot-test1.R. See also the paper for more details on templates.
Make sure that the previous steps (performing simulations and plotting) work. If they are not working properly, the run distribution will also fail! First, get the RSPLIB RSerPool implementation source package from here and its installation handbook from here. In the following, only a very brief installation guideline is given. Refer to the handbook for details on RSerPool. Furthermore, make sure that the lksctp-tools package (download here) is installed. Under Ubuntu/Debian Linux, you can download and install LKSCTP using the following command line:
sudo apt-get install libsctp-dev
Unpack the RSPLIB archive, then configure and compile it (in its directory):
./configure --enable-kernel-sctp make
Go to the rsplib/ directory and start a registrar:
./registrar
Start a scripting service pool element in another shell (the parameter -ssmaxthreads specifies the number of parallel sessions; use the number of cores/CPUs in your machine):
./server -scripting -policy=LeastUsed -ssmaxthreads=1
The output of "server" should look as follows:
Starting service ...
Scripting Server - Version 1.0
==============================
General Parameters:
Pool Handle = ScriptingPool
Reregistration Interval = 30000 [ms]
Runtime Limit = off
Max Threads = 2
Policy Settings
Policy Type = LeastUsed
Load Degradation = 0.000 [%]
Load DPF = 0.000 [%]
Weight = 0
Weight DPF = 0.000 [%]
Scripting Parameters:
Transmit Timeout = 60000 [ms]
Keep Temp Dirs = no
Registration
Identifier = $06355901
In particular, take care of the "Identifier" line. This is the ID of the pool element under which it has been registered. If there are error messages saying that registration has failed, etc., have a look into the RSPLIB handbook. Usually, this means a small configuration problem which can be solved easily!
Have a look into the script ssdistribute. Ensure that the variable setting for SIMULATION_POOLUSER points to the executable scriptingclient of the RSPLIB package. If scriptingclient is located unter src/rsplib-2.7.4/rsplib in your home directory, the line should be:
SIMULATION_POOLUSER=~/src/rsplib-2.7.4/rsplib/scriptingclient
In test1.R, set "distributionProcs" to the maximum number of simultaneous sessions (at least 1; if you later start 5 pool elements with 2 cores each, you should use 10). It is safe to use 1 for the following test. After modifying "distributionProcs", increase "simulationRuns" e.g. by 1. Otherwise, since you have already performed the run of test1.R before, no more runs would be necessary (since their results are already there!). Now, run test1.R again:
R --vanilla < test1.R
Have a look at the output of "server": it should receive jobs and process them. Also, have a look at the log output:
tail -f test1/make.log
When the job distribution is working properly, you can start more pool elements and set up your simulation computation pool. Do not forget to increase "distributionProcs" accordingly!
The workload distribution system works as follows:
The Makefile first generates a Tar/BZip2 file simulation-environment.tar.bz2 in the simulation directory. It contains the simulation binary, all shared libraries it needs (found out by the get-libs script) and the script simulation.config-stage0 which sets two environment variables: SIMULATION_PROGRAM contains the name of the binary, SIMULATION_LIBS contains the location of the libraries. If your simulation needs additional files (e.g. .ned files), they can be specified by the variable simulationMiscFiles in simulate-version1.R.
ssdistribute - which is called to actually distribute a run to a pool - creates the Tar/GZip file for the run. This file includes the environment file (i.e. usually simulation-environment.tar.bz2) specified by the variable SIMULATION_ENVIRONMENT and additional configuration files like simulation.config-stage0 (but named simulation.config-stage1, simulation.config-stage2, ...) specified by the environment variable SIMULATION_CONFIGS. You have to set these two environment variables in the ssdistribute script. Furthermore, the Tar/GZip file of the run contains the .ini file for the run.
ssrun performs a run on a (usually) remote node. First, it finds all simulation.config-stageX scripts and executes them in alphabetical order. That is, simulation.config-stage1 may overwrite settings of simulation.config-stage0 and so on. After that, it looks for .ini files. For each .ini file, it runs the program specified by the environemnt variable SIMULATION_PROGRAM. If the variable SIMULATION_LIBS is set, does not call the binary directly but tells the shared library loader to do this and use the specified set of shared libraries.
If everything went well, a status file is created. The existence of this status file means that the run has been successful.
Finding out what is going wrong with the remote execution can be difficult sometimes. In such a case, only start a single instance of "server" and use the parameter "-sskeeptempdirs". This parameter results in not deleting the temporary session directory after shutdown of the session. That is, you can dissect the directory's contents for troubleshooting. The name of the directory for each session is shown in the output of "server".
In order to use SimProcTC with your own model, perform the following tasks:
Before using the RSerPool-based run distribution, first test your simulation on your local machine! This makes finding problems much easier. If everything works, you can continue with run distribution.
If you discover problems or difficulties with the load distribution or RSPLIB, write to the RSPLIB General Discussion List. It is also strongly recommended to subscribe to the RSPLIB Announcement List:
Current version:
Old versions: