Sector Data File Formats

TRADER Format

NOTE: This format is documented for posterity; it is not supported by the site's tools.

The original electronic format for sector data was defined by Marc Miller in "Electronically Exploring the Traveller Universe" Challenge Magazine, Issue No. 26, pp.24-25. The sector generator program listing generated files matching this format. The format was also used in the TRADER game/utilty published by GDW for the Apple II computer system. The disk included SPINWARD and SOLOMANI data files for the two published sectors.

Notably absent from this early format is the world name, possibly for space reasons. The allegiance code precedes the base code, but already the format is easily recognizable. This format corresponds closely to the data from Classic Traveller publications such as Supplement 3: The Spinward Marches and Supplement 10: The Solomani Rim.

The format utilizes ASCII encoding; each character takes one byte and accented characters and non-Latin characters are not supported. Fields are fixed width, left aligned, and padded with spaces (ASCII 32). Records are also padded to ensure that each record occupies the same number of bytes; this allows programs to load specific records by index without loading the entire file, a critical requirement for the limited computer systems available at the time.

Following the introduction of this format, Traveller standard world data expanded to include several new fields such as "PBG" (population multiplier, number of planetoid belts, number of gas giants) and stellar data, culiminating in the data published in GDW's The Spinward Marches Campaign.

GEnie Format ("Sunbane")

Joe Fugate of Digest Group Publications generated data for the Imperium and several surrounding sectors, 35 in total. This data was made available to the public by uploading to the GEnie information service. It was later hosted on an FTP file server named "Sunbane". This dataset is often referred to as "GENII" (sic) or most commonly "the Sunbane data".

This format includes the PBG code (population multiplier, number of planetoid belts, number of gas giants), stellar data, and of course the mainworld name. Note that a long world name may be truncated, and there is not necessarily a blank between the name and hex location, and names may be truncated to fit. This format corresponds closely to that present in Classic Traveller, MegaTraveller, Traveller: The New Era and Marc Miller's Traveller (T4) publications such as The Spinward Marches Campaign, MegaTraveller Alien: Vilani & Vargr, The Regency Sourcebook (TNE) and Second Survey (T4).

This is a line-oriented, plain text, fixed-width format. The format utilizes the Windows-1252 encoding, so each character takes one byte and Western European accented characters are supported, but non-Latin characters are not. Lines may be terminated with LF (ASCII 10) or CR+LF (ASCII 13, ASCII 10). Each column has a fixed number of characters, padded with spaces (ASCII 32) as necessary. Trailing spaces at the end of a line may be truncated.

The file helpfully includes a self-documenting, multi-line header:

Spinward Marches
 1-13: Name
15-18: HexNbr
20-28: UWP
   31: Bases
33-47: Codes & Comments
   49: Zone
52-54: PBG
56-57: Allegiance
59-74: Stellar Data

....+....1....+....2....+....3....+....4....+....5....+....6....+....7....+....8

This is followed by system data:

Legacy SEC Format

As the introduction to this document alludes, the is no one true SEC file format. Various applications have taken canonical name/hex/uwp/bases/codes/zone/pbg/allegiance/stellar fields and applied various field widths. It is valuable to document a generalized version of this format which applications can consume.

This is a line-oriented, plain text, variable-width format. The format utilizes the Windows-1252 encoding, so each character takes one byte and Western European accented characters are supported, but non-Latin characters are not. Lines may be terminated with LF (ASCII 10) or CR+LF (ASCII 13, ASCII 10). Blanks may be spaces (ASCII 32) or tabs (ASCII 9).

To accomodate legacy files, this format does not define field widths. Instead, parsers must inspect lines using Regular Expressions (or equivalent logic) to identify the fields. The regular expressions here follow the Perl 5 extended regular expression syntax which has become a de-facto standard in many programming environments.

For each line in the file:

• Trailing whitespace on each line should be removed
• Blank lines should be ignored
• Lines starting with # (ASCII 35) should be ignored (comments in some applications)
• Lines starting with $ (ASCII 36) should be ignored (routes in some applications)
• Lines starting with @ (ASCII 64) should be ignored (subsectors in some applications)
• Lines matching the following regular expression produce the named fields.
• Other lines should be ignored. They may contain errors, or be comments.

^
( \s*       (?<Name>       .*                            ) )
( \s*       (?<Hex>        \d\d\d\d                      ) )
( \s{1,2}   (?<UWP>        [ABCDEX][0-9A-Z]{6}-[0-9A-Z]  ) )
( \s{1,2}   (?<Base>       [A-Z1-9* ]                    ) )
( \s{1,2}   (?<Remarks>    .{10,}?                       ) )
( \s+       (?<Zone>       [GARBFU]                      ) )?
( \s{1,2}   (?<PBG>        \d[0-9A-F][0-9A-F]            ) )
( \s{1,2}   (?<Allegiance> (\w\w\b|\w-|--)               ) )
( \s*       (?<Stars>      .*?                           ) )
\s*$

The complexity associated with the regular expression merits discussion:

• For simplicity, Traveller eHex codes are matched as [0-9A-Z]. More strictly, this should be: [0-9A-HJ-NP-Z]
• Name: Some files include + (ASCII 43) after the name to signify special characteristics. T4-era files terminate the name with . (ASCII 46). Some files use the hex location (e.g. 0133) as a placeholder when the name is not known. Applications should filter all of these.
• Base: Some files include * (ASCII 42)
• Remarks: A minimum width is imposed; otherwise a world with a base and no codes cannot be distinguished from a world with a zone and no codes.
• Zone: Includes additional codes:
  • A = Amber
  • R = Red
  • B = Traveller: The New Era Technologically Elevated Dictatorship (a.k.a. "Blue")
  • F = Forbidden (Zhodani)
  • U = Unabsorbed (Zhodani)
• PBG: P must be 0-9, but B and G can be eHex codes
• Many files fail to separate fields. It is common to "run together" the zone, PBG and allegiance codes (e.g. R456Im). (NOTE: The above regular expression does not account for this.)
• Stars: many stellar data formats are in use; some use additional glyphs to indicate the pair and near/far companions, as well as identifying the mainworld's star. Also, many files include non-stellar data as extensions. Additional parsing may be applied to distinguish these from stellar data, using the following regular expression:

^
(?<primary>
  ( [OBAFGKM] [0-9] ) \s* (D|Ia|Ib|II|III|IV|V|VI|VII) |
  (DB|DA|DF|DG|DK|DM|D) |
  Un
)

(?:
  \s+
  (?<companion>
    ( [OBAFGKM] [0-9] ) \s* (D|Ia|Ib|II|III|IV|V|VI|VII) |
    (DB|DA|DF|DG|DK|DM|D) |
    Un
  )
)*
$

From experience, files are "messy". Numerous files transcribed (by hand or via OCR) from print publications contain errors, such as I/1/l or 0/O. It is strongly encouraged that applications which consume legacy SEC files be made robust against such errors and report parse errors in a useful way to the users.

The following are some example legacy SEC data file entries found on the Internet, with comments:

# Note spacing between allegiance and stars, and "F" zone:
Iashplie      0701 B658452-C  Z Ni                 917 Zh     K9V M0D
Prianaf       1520 XA99000-0    Ba Lo Ni        F  003 Zh     F3V M7D

# Note placeholder world name:
0103               0103 X348002-0   Lo Ni             020 Na F7 V
Eahyo              0131 X433731-0   Na Po             400 As M7 V M9 D

# Note missing name, and "--" for allegiance, no stars:
                     0639 X440000-0   Ba Lo Ni Po De         013 --
Nnurukgr             0705 E637900-5   Hi In                  213 AW

# Note non-standard travel zones:
Elliador           1307 A67A3DD-B F Lo Ni Wa Cx Hw     O 724 Es M0 III
Cinhoss            1403 X8C06BF-2 M De Ni              H 620 Es M1 V

# Note "J-" for allegiance:
Kue Urzue          0207 C6568AB-5                   R 100 J- F2 V
Zimigsika          0209 A77399B-C J Hi In Cp          412 J- F0 V

# Note "." placeholder name:
.             3238 X610000-0    Ba Lo Ni           012 --     F4V
Rudzaghz      0139 D261610-2    Ni                 314 Va     G3V M3D

# Note placeholder names, no stars:
G-1           1715 A301735-9  N Ic Na Va           224 Na
G-2           1718 E7A0437-4    De Ni              604 Na

# Note wide fields, to accomodate long names:
Dujj't'kzo                0101 E331000-0    Ba Ni Po                        703 Ia M3 III M4 V
Hastitan                  0104 E460343-4    De Lo Ni Po                     103 Na M2 V M4 V

# Note possibly truncated world name:
Iyaaahai      2609 C7C05L6-9    De Ni              905 A5 M7 III M7 D
Eaaileishryaor2610 BAD57M6-E    Fl                 404 As M3 V

# Note world name terminated with ".":
Ihkasya            1020 B547100-6   Ni                603 Na
Ftihahe.           1023 D5A07M8-C   De                403 As

Despite the allowances in the de-facto format for variable width fields, it is strongly recommended that applications output individual sector files with fixed width columns. Care should be taken to avoid truncating any fields. A pre-processing step which determines the maximum length of the name and remarks field may be necessary prior to serializing any of the lines in the file. In addition, a minimum field width for name and remarks should be selected, both to aid in reading the file (for example, in the case where no systems are named) and to avoid collapsing the base, remarks, and zone columns.

It is also conventional to include a human-readable header which contains information about the file, including identifying the fields, identifying the sector, and possibly other information such as author, source, subsector names, etc.

T5 Tab Delimited Format

Traveller⁵ introduces several new fields beyond those in previous editions of Traveller, and are of course documented in the Traveller⁵ Core Rulebook. These include: Economic extension, Cultural extension, Importance extension, Nobility, and World count. In addition, the Traveller⁵ Second Survey (T5SS) is established to define a consistent, official data set for licensees and players. This allows us to establish a canonical data format.

The format utilizes the UTF-8 encoding. Therefore, each character may take more than one byte, but most of Earth's writing systems can be represented using Unicode. This is a line-oriented, plain text, tab-delimited format. Lines in the file are terminated with either LF (U+000A) or CR+LF (U+000D, U+000A). Every line in the file has fields separated by the Tab character (U+0009). There is no escaping mechanism – neither tabs nor line breaking characters may occur within fields.

Blank lines are ignored. Lines starting with # (U+0023) are ignored.

The first non-blank/non-comment line in the file is the header; it is tab-delimited and defines the fields. The fields do not appear in a fixed order. Parsers should read the header line and use it to construct a field index to field name mapping for use with the rest of the file.

Note that the "extension" fields (Ix, Ex, Cx) are given visually distinct wrapper characters. This is convention to distinguish the fields when data is in print format. Due to the complexity of parsing legacy SEC files, the use of tab-delimited format for T5 Second Survey data is strongly recommended.

Additional columns may be present, containing convenient but redundant data. These commonly include:

• Sector - sector abbreviation (4 letters)
• SS - subsector (A - P)
• RU - Resource Units (calculated from Economic Extension)

T5 Column Delimited Format

NOTE: If not explicitly specified by request headers, the site's tools only auto-detect this format if the {Ix}, (Ex) and [Cx] headers are present. Otherwise another format will be inferred. Include those fields in the header (even if the content is blank) to use this format.

Although the tab-delimited format is the most convenient for interchange it is difficult for humans to read. This document establishes an extensible, human-readable plain text format for the Traveller⁵ Second Survey data.

The format utilizes the UTF-8 encoding. Therefore, each character may take more than one byte, but most of Earth's writing systems can be represented using Unicode. This is a variable-width, line-oriented format. Fields are fixed width, separated by at least one space, left aligned and padded by spaces. Lines in the file are terminated with either LF (U+000A) or CR+LF (U+000D, U+000A). There is no escaping mechanism – neither tabs nor line breaking characters may occur within fields.

NOTE: At this time, Unicode code points beyond U+FFFF are not correctly supported by the site. This is because the underlying implementation internally uses .NET's UTF-16 representation for strings, and characters beyond U+FFFF encode into two UTF-16 code units, which mean column counts will be incorrect.

Blank lines are ignored. Lines starting with # (U+0023) are ignored.

The first non-ignored line in the file is the header line. It establishes the field names. The header line is composed of field names padded by spaces. The header line is followed by a separator line which defines the field widths. The separator line is composed of runs of - (U+002D) characters separated by one or more space characters (U+0020). To parse the file, the header line should be saved, then the separator line parsed to determine field widths and offsets. The header line can then be parsed to determine field names. The fields do not appear in a fixed order; after parsing the separator line and the header line, a field index to field name mapping can be constructed to use for parsing the rest of the file.

Leading and trailing spaces on the separator line are permitted. Non-space characters that appear in any line between fields result in a parse error.

The rest of the non-ignored lines in the file are data lines and are parsed using the fields and widths determined by parsing the header line.

The following is an example header line, separator line, and data line:

The field definitions are the same as those given for the T5 Tab Delimited Format above, and so are not repeated here. While the field order should not be assumed by applications parsing this format, applications should follow the Traveller⁵ Second Survey standard defined in the Traveller⁵ Core Rulebook: Hex, Name, UWP, Remarks, Ix, Ex, Cx, N, B, Z, PBG, W, A, Stellar.