copy command in postgresql to append data

Assuming a Unix system

with PostgreQSL 9.3 or newer

Use the PROGRAM clause with cat as the program.

execute 'copy (select * from tableName) to PROGRAM ''cat >>/tmp/result.txt'''

This would append the results to the end of /tmp/result.txt instead of overwriting the file.

with older versions of PostgreSQL

A named pipe can be used by a separate process to concatenate results. Example:

$ mkfifo /tmp/pgfifo
$ while true; do cat /tmp/pgfifo >>/tmp/result.txt; done

This last command will block. Let it run until all the results are accumulated, then it can be terminated with ^C or kill.

In SQL, the fifo has to be fed with:

execute 'copy (select * from tableName) to ''/tmp/pgfifo'''

I think you cannot (easily) do that (at least on your PostgreSQL version, see Daniel's answer). What you definitely can, however, is changing your approach, and do a UNION ALL from all those tables in the query part of your COPY. This means there is no looping, but you have to construct your query from the collected table names.

The result would look like

COPY (SELECT * FROM table1
      UNION ALL
      SELECT * FROM table2
      ...) 
    TO tmp/result.txt;

Notes:

  • there is no such thing as a psql function. psql is a client to PostgreSQL. It has built in commands (basically everything starting with \), however.
  • it is not so clear what you mean by 'archive logs'. If it is the write-ahead log (WAL), then with the above approach you don't have to worry about it. Otherwise, you can use unlogged tables.
  • if the tables are big, there is a chance your system will write (a lot of) temp files. This might make the execution not so fast.