This week covers June 17th thru June 21st.
2.1. General Notes
During this week, I also set up my Google Calender, linking it with Iwona’s and then adding the appropriate email(redacted for blog. Contact if needed) in the same way as Iwona’s calendar was added, in order to get access to NERSC events. I also set up LaTex syntax highlighting and finished stuff such as my CalID.
2.2. Goals
The goals this week were to do the PROOF benchmark on PDSF and HOPPER. PDSF was completed, HOPPER had some errors. I also continued documentation in LaTex.
2.3. Syntax Highlighting in LaTex
TexWorks, as it was installed automatically on macs, was used. In order to set it up, you must go to ~/Library/TeXworks/ and then go to configuration. There, you should go to the Syntax Patterns file, opening with emacs or like, and set the LINUX settings to Y, saving the file. Then, go to texworks:texworks/preferences and set the Syntax coloring to LaTex. Then, go to the GUI of your editor, opening your document, and going to format/syntax coloring and click on LaTex as your choice.
2.4. The process of Benchmarks
In order to accurately gauge the efficiency of ROOT on a system with optional parallelization, one uses the TProof benchmarks test to check the CPU and IO efficiency. It is described in detail at http://root.cern.ch/drupal/content/proof-benchmark- framework-tproofbench
It should be noted that CPUx and RunDataSetx in the benchmarks required multiple nodes to run on, and so are not presently run.
PDSF In order to accurately gauge efficiency, one must know where to start from. Thus, the benchmark test was run on PDSF.
The following macro enabled one to run the benchmarks in root, either via .x proofBenchData.c or by running it via a shell script. The same commands do work interactively.
//proof2.c
void proof2(){
TProofBench pb("");
pb.MakeDataSet();
pb.RunDataSet();
}
It produces the following image, among others, which shows the efficiency. This particular image does not normalize the results, which will be important in making comparisons later with the Hopper results.
The following macro does the same as the DataSet benchmarks, but with the CPU benchmark.
#include <iostream>
using namespace std;
//proof.c
void proof(){
gEnv->SetValue("ProofLite.Sandbox", "/location/of/sandbox");
TProofBench pb("");
pb.MakeDataSet();
pb.RunDataSet();
}
It produces the following image, among others.
2.5. Errors and Resolution
There were various errors in getting the benchmarks to run.
Path errors
Some of the errors came from setting paths incorrectly, resulting in the following commands needing to be run at startup. Path setting is important because it tells the computer where to look for the libraries and binaries/executables. It is important to tell the computer to look for the ROOT modules, as opposed to the linux system ones. The same can be said of Hopper, but with the directories changed to the correct ones, of course.
. bin/thisroot.sh export LD_LIBRARY_PATH= /global/project/projectdirs/pdsf/amct/pdsf/root/lib/: /common/nsg/sge/ge-8.1.2/lib/lx-amd64:/common/usg/lib/PATH=$PATH:/global/project/projectdirs/pdsf/amct/pdsf/root
CPUx and DataSetx
Another error was that if RunCPUx or RunDataSetx were run, as was done with RunCPU and RunDataSet above, segfaults abounded! This turned out to be due to the fact I was running on a single node, not multiple. The x suffix measures the numbers of workers as the nodes are increased, not just as one increases multithreading on the processor’s cores itself.
Proof Master
Another question that resulted in errors was a question of where the PROOF master for the benchmarks was. It turns out that you can leave it blank, and declare a benchmark as above, with a TProofBench pb(“”), i.e, empty constructor, as opposed to explicitly stating it.
Assorted minor errors
Other frequent errors included misspelling names of files, and thus them not being found; paths not being found, fixed by explicitly setting the path to the macro in the file, from global, not from the current directory; libraries not being found, fixed by setting a direct path to bin/thisroot.sh
Memory errors ROOT needed more memory, which was resolved by doing the top command to see how much memory was needed while running interactively and then modifying qsub with the flag -l h_vmem=xG where x is the amount of memory needed, in our case, around 10, in GB
Quota errors Quotas were often overloaded, which was fixed by running du and deleting the largest useless file with rm, i.e, all the data PROOF creates on the data benchmark, in the ./.proof file. This could also be resolved by changing the working directory, and the submission rules can be snuck around by submitting from pdsfgrid.
Data set issues, the beginning There were assorted issues with HOPPER not producing a dataset, on the data benchmark, either not finding me as a user, or not finding the dataset, this was resolved by modifying the aprun command to be as follows:
aprun -n 1 -N 1 -d 24 -cc none sh /global/project/projectdirs/pdsf/amct/hopper /root/rootShell2.sh
These options connate the number of processors to use at first, followed by the - d flag being used for the max depth. They are described more in the man page. It also turned out that due to the version of Linux that HOPPER runs being limited, that a full BASH shell was not supported. Thus, a sh in aprun was used instead of bash, without success. It was resolved in week 3.
The various errors were resolved on PDSF, and week 3 was left to resolve HOPPER issues. The CPU Benchmark worked on HOPPER and will be described in more detail when it has been resolved.
amctaggart 