4.1 CFITSIO Size Limitations

CFITSIO places few restrictions on the size of FITS files that it reads or writes. There are a few limits, however, which may affect some extreme cases:

1. The maximum number of FITS files that may be simultaneously opened by CFITSIO is set by NMAXFILES, as defined in fitsio2.h. The current default value is 1000, but this may be increased if necessary. Note that CFITSIO allocates NIOBUF * 2880 bytes of I/O buffer space for each file that is opened. The default value of NIOBUF is 40 (defined in fitsio.h), so this amounts to more than 115K of memory for each opened file (or 115 MB for 1000 opened files). Note that the underlying operating system, may have a lower limit on the number of files that can be opened simultaneously.

2. By default, CFITSIO can handle FITS files up to 2.1 GB in size (2**31 bytes). This file size limit is often imposed by 32-bit operating systems. More recently, as 64-bit operating systems become more common, an industry-wide standard (at least on Unix systems) has been developed to support larger sized files (see http://ftp.sas.com/standards/large.file/). Starting with version 2.1 of CFITSIO, larger FITS files up to 6 terabytes in size may be read and written on supported platforms. In order to support these larger files, CFITSIO must be compiled with the '-D_LARGEFILE_SOURCE' and `-D_FILE_OFFSET_BITS=64' compiler flags. Some platforms may also require the `-D_LARGE_FILES' compiler flag. This causes the compiler to allocate 8-bytes instead of 4-bytes for the `off_t' datatype which is used to store file offset positions. It appears that in most cases it is not necessary to also include these compiler flags when compiling programs that link to the CFITSIO library.

If CFITSIO is compiled with the -D_LARGEFILE_SOURCE and -D_FILE_OFFSET_BITS=64 flags on a platform that supports large files, then it can read and write FITS files that contain up to 2**31 2880-byte FITS records, or approximately 6 terabytes in size. It is still required that the value of the NAXISn and PCOUNT keywords in each extension be within the range of a signed 4-byte integer (max value = 2,147,483,648). Thus, each dimension of an image (given by the NAXISn keywords), the total width of a table (NAXIS1 keyword), the number of rows in a table (NAXIS2 keyword), and the total size of the variable-length array heap in binary tables (PCOUNT keyword) must be less than this limit.

Currently, support for large files within CFITSIO has been tested on the Linux, Solaris, and IBM AIX operating systems.