Condor-G Example7 - Run A DAGMan Example
- DAG - Directed Acyclic Graph
- Programs are interdependent and must run in a certain order
- Programs are Nodes on Graph
- Edges Identify the Dependencies
- DAGMan - Directed Acyclic Graph Manager
There is a DAGMAN submit file called mydag.dag which contains the following lines and describes
a DAG :
Job Setup setup.submit
Job WorkerNode_1 run1.submit
Job WorkerNode_2 run2.submit
Job WorkerNode_3 run3.submit
Job Finalize finalize.submit
PARENT Setup CHILD WorkerNode_1 WorkerNode_2 WorkerNode_3
PARENT WorkerNode_1 WorkerNode_2 WorkerNode_3 CHILD Finalize
There are Condor submit files for each part of the DAG. The file setup.submit looks like this:
executable = setup
Transfer_Executable = true
globusscheduler = beowulf2.dartmouth.edu/jobmanager
universe = grid
output = setup.out
error=setup.error
log = setup.log
queue
Create run1.submit, run2.submit, run3.submit and finalize.submit. They will look like setup.submit
except they will replace submit by run1, run2, etc. Here is what run1.submit will look like:
executable = run1
Transfer_Executable = true
globusscheduler = beowulf2.dartmouth.edu/jobmanager
universe = grid
output = run1.out
error=run1.error
log = run1log
queue
sierra:~> more setup.submit
executable = setup
Transfer_Executable = true
globusscheduler = beowulf2.dartmouth.edu/jobmanager
universe = globus
output = setup.out
error=setup.error
log = setup.log
queue
sierra:~>condor_submit_dag mydag.dag
sierra:/scratch/sas/VDT/DAG_example> condor_q
-- Submitter: sierra.dartmouth.edu : <129.170.16.93:37147> : sierra.dartmouth.edu
ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD
74.0 sas 5/3 15:36 0+00:00:26 R 0 2.6 condor_dagman -f -
75.0 sas 5/3 15:36 0+00:00:01 C 0 0.0 setup
1 jobs; 0 idle, 1 running, 0 held
sierra:/scratch/sas/VDT/DAG_example> condor_q
-- Submitter: sierra.dartmouth.edu : <129.170.16.93:37147> : sierra.dartmouth.edu
ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD
74.0 sas 5/3 15:36 0+00:00:36 R 0 2.6 condor_dagman -f -
76.0 sas 5/3 15:37 0+00:00:00 I 0 0.0 run1
77.0 sas 5/3 15:37 0+00:00:00 I 0 0.0 run2
78.0 sas 5/3 15:37 0+00:00:00 I 0 0.0 run3
sierra:/scratch/sas/VDT/DAG_example> condor_q
-- Submitter: sierra.dartmouth.edu : <129.170.16.93:37147> : sierra.dartmouth.edu
ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD
74.0 sas 5/3 15:36 0+00:01:04 R 0 2.6 condor_dagman -f -
76.0 sas 5/3 15:37 0+00:00:01 C 0 0.0 run1
77.0 sas 5/3 15:37 0+00:00:00 C 0 0.0 run2
78.0 sas 5/3 15:37 0+00:00:01 C 0 0.0 run3
79.0 sas 5/3 15:37 0+00:00:00 I 0 0.0 finalize
sierra:/scratch/sas/VDT/DAG_example> condor_q
-- Submitter: sierra.dartmouth.edu : <129.170.16.93:37147> : sierra.dartmouth.edu
ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD
76.0 sas 5/3 15:37 0+00:00:01 C 0 0.0 run1
77.0 sas 5/3 15:37 0+00:00:00 C 0 0.0 run2
78.0 sas 5/3 15:37 0+00:00:01 C 0 0.0 run3
79.0 sas 5/3 15:37 0+00:00:00 C 0 0.0 finalize
0 jobs; 0 idle, 0 running, 0 held
sierra:/scratch/sas/VDT/DAG_example> more setup.out
Program /home/grid1/.globus/.gass_cache/local/md5/96/207e9f53e87feadf3a56685588c998/md5/6d/d401fa98af43ef66bce4d45d7b47dc/data
running on beowulf2 at Tue May 3 15:37:07 2005.
sierra:/scratch/sas/VDT/DAG_example> more run1.out
Program /home/grid1/.globus/.gass_cache/local/md5/09/8dd5cf2546294a0db47f933a21f7a4/md5/65/519da3da90171ce2cb1edb2ba85ac9/data
running on beowulf2 at Tue May 3 15:37:33 2005.
