The MegaPipe image stacking pipeline

The MegaPipe image stacking pipeline CFHTLS catalogue query documentation

CFHTLS catalogue query documentation

Setting the query: Use the check boxes to select which columns to return. Apply constraints on any column using the entry boxes. Use the positional constraint boxes if you want restrict the search to a smaller area. Use magnitude limit, masking and field criteria. It's usually best to test a query with "Max rows" set to a small number, then switch to unlimited rows once you're sure you've got the constraints right.

Synchronous vs. Asynchronous queries: For small queries, restricted in position, only returning a hundred thousand rows or less, synchronous queries are fine. Just hit submit and wait a few seconds for the results. The results will show up directly in your browser. However, the CFHTLS database contains 30 million sources with 196 parameters recorded for each source; if your query will return a significant fraction of the database, the query will take up to an hour to complete. The connection between your broswer and the CADC server will probably be lost. In this case, you are probably better off using the asynchronous queries. In this the case, your query will be executed as before but the webpage returned will just indicate that the query is running until it completes, when it will give you a link to the location of the results in the cfhtlsmegapipe queries VOspace. The results file will perist there for at least a week. Note that for asynchronous queries, the only return formats available are TSV, CSV, and VOTables. If you lose track of your query, it will end up cfhtlsmegapipequeries VOspace with a file name that is a combination of "cfhtlsmegapipe" and the time (UTC) when you submitted the query. For example: cfhtlsmegapipe_2012-07-12T20:38:03.tsv

Setting the query: As you fill out the query page form, you will notice that the text in the query box changes. You can also modify the text directly for a higher degree of customization. As well as selecting individual columns you can select combinations of columns: G_MAG_AUTO-R_MAG_AUTO for example. The query language is the Astronomical Data Query Language or ADQL which was developed by the IVOA. It's very similiar to SQL with a few additions. The only caveat concerning setting the query manually is that if you modify the query manually first, and then use the various checkboxes and fields on the web page, it will overwrite your edits.

Executing multiple queries: If you click submit from the web page, you should get back the results. For one-off queries, this is fine. However if you want to execute multiple (small) queries, you may want to execute the queries from a script. You can modify the query by changing the part of the URL after "query=" to set an arbitrary query (remember to encode the special characters). In synchronous mode, GETs to that URL should work.

For example:

  wget -O test.txt ''

Or, if you want to go directly to the TAP server, you could write:

  wget -O test.tsv ''

Note that the TAP server only supports the TSV, CSV and VOtable formats.

Behind the scenes

Table Access Protocol (TAP): This is the VO standard for accessing tables. The CFHTLS MegaPipe catalogue query page is running TAP for you behind the scenes. However feel free to bypass them and access the tables directly. Documention for TAP in general can be found on the IVOA TAP documentation page.

Astronomical Data Query Language (ADQL): The query language used by TAP is ADQL. As mentioned before, ADQL is based on SQL, special restrictions and extensions to SQL92 have been defined in order to support generic and astronomy specific operations. The full details of ADQL can be found on the IVOA ADQL documentation page.

Universal Worker Service (UWS): In asynchronous mode, TAP uses UWS to monitor the progress of the query. More documentation for UWS is available on the IVOA UWS documentation page