Table Of Contents

Previous topic

LDAC - Advanced Catalogue Manipulations

Next topic

LDAC - Some Useful Tasks

This Page

LDAC - Catalogue Matching


The concrete examples on this page use the LDAC catalogue, the ASCII tables W3m0m2_specz_1.asc, W3m0m2_specz_2.asc and the three configuration files asctoldac_specz.conf, associate_photoz_specz.conf and make_ssc_photoz_specz.conf

A tar archive with the complete set of LDAC Tools example files is here

We often need to merge astronomical catalogues from different sources to compare object properties or to merge quantities distributed in different object catalogues. The LDAC tools provide the program pair associate and make_ssc to perform these tasks with a large amount of options. We cover here only the most basic application by associating three catalogues and creating a merged one. For a more complete description of the available options consult Chapter 5 of Erik Deuls Pipeline Documentation

LDAC Catalogue Matching - associate

The object matching process implemented within the LDAC tools is a two-stage process. First, the program associate matches sources in different catalogues (we neglect here the possibility of matches within the same catalogue). The extent of each source is characterised by an (error) ellipse in a two-dimensional position space, for instance sky-coordinates with Ra and Dec or pixel coordinates with X and Y. To successfully run associate a minimum set of parameters needs to be present for each object in each catalogue that participates in the matching process. The naming conventions in the following table list parameter names of SExtractor. However, the actually used names can differ and are subject to settings in a configuration file. We list possible parameters for a matching in Sky / Pixel coordinates:

Parameter SKY / PIXEL Meaning for Object Association
SeqNr Unique Object Identification Number
(ALPHA_J2000, DELTA_J2000) / (X_IMAGE, Y_IMAGE) Object Ellipse Center
(A_WORLD, B_WORLD) / (A_IMAGE, B_IMAGE) Semi major and Semi minor ellipse axis
THETA_WORLD / THETA_IMAGE position angle of semi major axis
Flag Extraction - Quality Flags of the object

The associate program establishes an association between objects in different catalogues if the center position of one object lies within the ellipse spanned by the other. The following figure illustrates several association possibilities - it is taken from Erik Deuls Pipeline Documentation

Object merging with associate

In the following examples we have an LDAC catalogue which lists galaxy properties, especially photometric redshifts, in a one square degree area of sky. In addition, we have spectroscopic redshifts from the same sky-area from two different sources. Those are listed in ASCII tables W3m0m2_specz_1.asc and W3m0m2_specz_2.asc. Our final goal is to compare the estimated photometric redshifts with their spectroscopic cousins and we need to merge the three catalogues. The first step is to convert the ASCII tables into the LDAC format and then to perform object association. Note that we need to give error ellipses, sequence numbers and a flag key to the obejcts in the ASCII tables. For the ellipses we consider circles with a radius of half an arcsecond and the quality flags are set to zero (object is o.k.) everywhere. We perform this initial modification of the ASCII catalogues with a simple awk command:

user$ less W3m0m2_specz_1.asc
213.8106384 51.94233322 0.2906414866
213.8327332 51.95418167 0.9577051401
213.8175049 51.95326233 0.9090785980


user$ awk '{printf("%d %15.8f %15.8f %15.8f %15.8f %f %d %f\n",
      NR, $1, $2, 0.5 / 3600., 0.5 / 3600., 0., 0, $3)}' W3m0m2_specz_1.asc >\

user$ asctoldac -i tmp.asc -o -t OBJECTS -c asctoldac_specz.conf

user$ awk '{printf("%d %15.8f %15.8f %15.8f %15.8f %f %d %f\n",
      NR, $1, $2, 0.5 / 3600., 0.5 / 3600., 0., 0, $3)}' W3m0m2_specz_2.asc >\

user$ asctoldac -i tmp.asc -o -t OBJECTS -c asctoldac_specz.conf

user$ rm tmp.asc

user$ associate
Object Association Program Version: 1.3.4 (Aug 24 2010)
(Pipeline Global Version: 1.2.0)

Usage: associate
              -i incat1 [incat2 [incat3 [...]]]       (input catalogs)
              -o outcat1 [outcat2 [outcat3 [...]]]    (output catalogs)
              [-t table_name]                         (table name of objects)
              [-c conf_file]
              [-OPTION1 OPTVAL1 [-OPTION OPTVAL2] ...]

user$ associate -i \
                -o \
                -t OBJECTS -c associate_photoz_specz.conf

Object Association Program Version: 1.3.4 (Aug 24 2010)
(Pipeline Global Version: 1.2.0)
Reading file
Reading file


  1. All catalogues that participate in an association process must carry the object properties in tables with the same names and in keys with the same names and the the same keytypes
  2. The configuration file associate_photoz_specz.conf lists the keys which hold the ellipse parameters. In addition, there is the possibility to exclude problematic sources, characterised by certain values of Flag, from the association process. The parameter INTER_COLOR_TOL is a scaling factor. The object ellipse parameters representing the semi-minor and semi-major axis are scaled with that value before the association takes place. Hence, it is easy to redo the association with different positional uncertainties.
  3. You can simultaneously associate up to 255 catalogues.


  1. If association should happen in pixel coordinates, the parameter -p must be given to the associate program. associate will NOT issue warnings or errors if you forget this parameter!
  2. You must ensure that all involved association parameters (ALPHA_J2000 etc.) are of the same keytype in all involved catalogues. This is avery common source for difficult to spot errors!

The associate program creates for each input catalogue a corresponding output catalogue. Note that the order is important, i.e. the first input catalogue is linked to the first output catalogue and so on. :

user$ ldactoasc -i -t OBJECTS -k SeqNr Pair_0 Pair_1 Pair_2
#   1 SeqNr               Running object number
#   2 Pair_0              Pairing with catalog dummy
#   3 Pair_1              Pairing with catalog Unknown
#   4 Pair_2              Pairing with catalog Unknown
        35 0 0 0
        61 0 487 0
      2775 0 0 114
     84164 0 4773 72

associate adds Pair keys to the objects tables which carry the information on the matching. Note that the counting starts at zero for the first catalogue. The above outputs reads as follows:

  1. Object 35 in the first catalogue has no matches with objects in other catalogues
  2. Object 61 in the first catalogue has a match with object 487 from the second catalogue
  3. Object 2775 from the first catalogue has a match with object 114 from the third catalogue
  4. Object 84164 from the first catalogue has a match with obejct 4773 from the second catalogue and a match with object 72 from the third catalogue


A number unequal to zero in the Pair_0 column of would mean that an object of has a match within that catalogue (close pairs)!

Correspondingly for the and

user$ ldactoasc -i -t OBJECTS -k SeqNr Pair_0 Pair_1 Pair_2
#   1 SeqNr               Object Sequence Number                          [""]
#   2 Pair_0              Pairing with catalog dummy
#   3 Pair_1              Pairing with catalog Unknown
#   4 Pair_2              Pairing with catalog Unknown
        1 0 0 0
      526 587 0 0
      527 0 0 0
      528 0 0 0
      529 642 0 0
  1. Object 526 in the second catalogue has a match with object 587 from the first catalogue
  2. Objects 527 and 528 from the second catalogue have no matches with obejcts in other catalogues
  3. Object 529 in the second catalogue has a match with object 642 from the first catalogue

The information of the Pair keys can be used with the program make_ssc to create a catalogue with merged object information.

LDAC Catalogue Matching - make_ssc

The make_ssc uses the Pairs information from associate to create a merged catalogue:

user$ less  make_ssc_photoz_specz.conf


user$ make_ssc
 Source Catalog Production Program Version: 2.0.4 (Aug 24 2010)

Usage: make_ssc
              -i incat1 [incat2 [..]] (input catalogs)
              -o outcat               (output catalog)
              [-t table_name]         (input catalog tablename to merge)
              [-p table_name catalog_nr [table_name catalog_nr ...]]
                                      (tables to copy from input to output)
              [-c config_file]
              [-OPTION1 OPTVAL1 [-OPTION OPTVAL2] ...]

user$ make_ssc -i -o \
               -c make_ssc_photoz_specz.conf

 Source Catalog Production Program Version: 2.0.4 (Aug 24 2010)
Indexing channel 2
WARNING(ssc_put_row): No key Ra in table PSSC
Objects read: 53248, written: 49195

user$ ldacdesc -i

******        Table #2
      Extension type:.........BINTABLE
      Extension name:.........PSSC

user$ ldacdesc -i -t PSSC
      Key name:...............ALPHA_J2000
      Key name:...............DELTA_J2000
      Key name:...............Z_B
      Key name:...............Z_SPEC_1
      Key name:...............Z_SPEC_2
      Key name:...............N_00
      Key name:...............N_01
      Key name:...............N_02
      Key name:...............SeqNr

make_ssc creates a table PSSC with the merged information. The configuration file make_ssc_photoz_specz.conf for make_ssc consists of blocks that determine which quantities from the initial catalogue are transfered how to the merged one:

COL_NAME = Z_SPEC_1 There will be a key Z_SPEC_1 in PSSC
COL_INPUT = Z_SPEC The original key has the name Z_SPEC
COL_MERGE = AVE_REG This line is always AVE_REG for the purpose discussed here
COL_CHAN = 1 The input key comes from the second input catalogue (counting starts with 0 for the first catalogue)


make_ssc offers a rich set of further options in the merging process. Please consult Erik Deuls Pipeline Documentation for thses possibilities.


As associate, make_ssc often produces difficult to find errors if keys with the same name in different input catalogues DO NOT have the same keytype!

The keys N_XX in the PSSC table give the number of obejcts that participate from the corresponding input catalogue (counting starts at zero!) in that entry:

user$ ldactoasc -i -t PSSC
#   1 ALPHA_J2000         Right ascension of barycenter (J2000)           [deg]
#   2 DELTA_J2000         Declination of barycenter (J2000)               [deg]
#   3 Z_B                 ""                                              [""]
#   4 Z_SPEC_1            spectroscopic redshift                          [""]
#   5 Z_SPEC_2            spectroscopic redshift                          [""]
#   6 N_00                Number of objects merged for channel 0
#   7 N_01                Number of objects merged for channel 1
#   8 N_02                Number of objects merged for channel 2
#   9 SeqNr               Object sequence number (from make_ssc)
214.2449074 52.12472016 0.7200000000 0.00000 0.00000 1 0 0 1
214.2613235 52.12476932 0.4800000000 0.00000 0.00000 1 0 0 2
214.5122864 52.12514428 0.8800000000 0.00000 0.00000 1 0 0 3
214.2435491 52.12493391 0.5200000000 0.00000 0.00000 1 0 0 4
213.8923482 52.12469253 0.8000000000 0.765736 0.00000 1 1 0 15
0.000000000 0.000000000 0.000000000 0.00000 0.132505 0 0 1 49747
0.000000000 0.000000000 0.000000000 0.00000 0.132391 0 0 1 49748

In the first lines, only the first object catalogue contributes. Hence, there are entries fro the coordinates and the photometric redshift Z_B. As no object from the second and third catalogue contributes the columns for Z_SPEC_1 and Z_SPEC_2 are filled with zero. Correspondingly, if no object from the first catalogue contrbutes, the coordinates and Z_B entries are set to zero.

The N_XX keys allows us now an easy filtering of important information: We plot comparisons of photometric redshifts with their spectroscopic counterparts:

user$ ldacfilter -i -o -t PSSC -c "(N_00=1)AND(N_01=1);"

user$ ldacfilter -i -o -t PSSC -c "(N_00=1)AND(N_02=1);"

user$ -i -t PSSC -k Z_B Z_SPEC_1 -l 0 1.5 0 1.5 \
                  -ti "Z_B vs. Z_SPEC_1" -o z_b_z_spec_1.png

Saving plot to z_b_z_spec_1.png

user$ -i -t PSSC -k Z_B Z_SPEC_1 -l 0 0.5 0 0.5 \
                  -ti "Z_B vs. Z_SPEC_2" -o z_b_z_spec_2.png

Saving plot to z_b_z_spec_2.png
Z_B Z_SPEC_1 comparison Z_B Z_SPEC_2 comparison