Skip to main content
A Chakra mapping file tells the simulator which server runs each Chakra rank. Without one, ranks are placed in order: rank 0 on the first Chakra-capable server, rank 1 on the second, and so on. With one, you choose the placement, and you can declare scale-up groups so that traffic inside a group uses the scale-up fabric instead of the scale-out network. You upload a mapping file once and attach it to a simulation configuration, and every simulation created from that configuration is launched with it. The file is stored byte for byte: a download returns exactly the bytes you uploaded.

Current limitations

A simulation created from a configuration with a mapping file attached runs to completion on a traceset you upload, and records the file in appliedMappingFile. These limits remain:
  • Generating a mapping file. The platform does not generate a mapping file for a configuration. Write one yourself, or generate one with the mapping file toolkit Scala provides (see Generate a mapping file), and upload it.
  • Tracesets you upload need a rank count. Attaching a mapping file requires the configuration’s traceset to have a rank count. A traceset you upload reports its rank count as rankCountDerived on GET /api/v1/tracesets/{id}; when that field is present, the traceset has a rank count. An upload of trace files declares its rank count in workloadMetadata, as shape.rankCount or as parallelism degrees (dp, tp, sp, ep, pp, multiplied together, with an absent degree counting as 1), and POST /api/v1/tracesets/upload-url refuses one that declares neither. A traceset uploaded before rank counts were declared may report no rank count. Behind an uploaded traceset with no rank count, the mapping-file attach answers 422 UNPROCESSABLE_ENTITY, and retrying does not change the answer: upload the traces again with a declaration. The traceset attach refuses the same traceset with the same 422, although a configuration create or update can still name it as activeTraceset.
  • The platform’s own Chakra tracesets (coming soon). These have a rank count, so a mapping file attaches to a configuration that uses one. A simulation created from that configuration currently fails before the simulator starts, with the failureReason Mapping file <name> (<id>) could not be delivered, until an update to the platform’s simulation hosts is in service.
The rest of this page describes the file format and the full flow as the API defines it.

The file format

A mapping file is plain text with one required section and one optional section.
  • Everything from a # to the end of its line is a comment. Blank lines are ignored.
  • Fields are separated by spaces, commas, or any run of both. Tabs are not separators, and the file must use Unix (LF) line endings, not Windows (CRLF).
  • A line may carry at most 127 bytes before its first comment, and at most 4,096 bytes in total.
  • A comment block at the top of the file must use #. A line starting with @ is reserved for Section II.

Section I: rank placement

Section I lists one rank per line, with exactly two or exactly four fields:
All fields are non-negative base-10 integers that fit in 32 bits, and 4294967295 is not accepted as a serverId. The ids must cover the contiguous range 0 through rankCount - 1, each exactly once, and no two ranks may name the same serverId. A rank with no scale-up fields belongs to no group, and all of its traffic uses the scale-out network.

Section II: scale-up group parameters (optional)

Section II begins at the first line that starts with @, after at least one Section I entry. Every line starting with @ is a comment, so the marker line can carry a heading, and inside Section II both # and @ start a comment anywhere in a line. Each remaining line declares one group, with four fields or six fields:
A line with four fields is an untyped group. An untyped group is an EPS group with one plane. A line with six fields is a typed group. A line with five fields, or with more than six fields, is refused. The bandwidth in bit/s is ScaleUpBandwidthGbps × 1e9, truncated to a whole number. For a typed group it must divide by NumOfPlanes with no remainder: 100 Gbit/s on 3 planes is refused, and 99 Gbit/s on 3 planes is accepted. On an OCS-XBar group, ScaleUpPerHopSwitchLatencyNs is the latency of one pass through the crossbar. A typed group (EPS or OCS-XBar, with any NumOfPlanes, including 1) needs a scale_up_plane attribute on every scale-up SEND and RECV node of the workload. A scale-up node is a node whose partner is in the same scale-up group. The attribute selects the plane of the transfer, and its value is an unsigned integer from 1 to NumOfPlanes. In the Chakra trace, scale_up_plane is an entry in the node’s attr list. The entry has name set to scale_up_plane, and its value is a JSON integer under the key uint64Val. The value is not a string. A string under uint64Val (the form comm_size uses) stops the run, and so does a value under any other key. For plane 2, the entry is:
The run stops when a scale-up node of a typed group:
  • has no scale_up_plane attribute,
  • has a value that is not a JSON integer under uint64Val, or
  • has a value outside 1 to NumOfPlanes.
An untyped group needs no scale_up_plane attribute, and it ignores one. The two latency fields are also bounded as a pair, using the group’s switch latency s and cable latency c, both in picoseconds:
  • The smallest crossing, 1 × s + 2 × c (one switch hop and two cable segments, the crossing between two ranks of the same subgroup), is at least 1 ps. A group with both latencies at 0 is refused. A group with one latency at 0 and the other positive is accepted.
  • The largest round trip, 2 × (3 × s + 4 × c), is not more than 9223372036854775807 ps, the largest time the simulator represents. A switch latency of about 1.5e15 ns, or a cable latency of about 1.15e15 ns, reaches this limit on its own, well below the per-field ceiling in the table.
The per-field rule still applies on its own: a switch latency of 0.0005 ns with a cable latency of 0.001 ns meets the pair bound, but the switch latency is below 1e-3 and is refused. Every group a Section I entry names must have a Section II line. Hexadecimal and non-finite values are not accepted. If the file has no scale-up groups, leave Section II out.

Examples

An 8-rank file with no scale-up groups, placing the ranks in reverse order. The comment on each row is free-form; here it records where the server sits in the topology:
An 8-rank file with two scale-up groups. Ranks 0 to 5 are group members; ranks 6 and 7 are not:
A 4-rank file with one typed group. Group 0 is an OCS-XBar group of 8 planes. Its switch latency is 0, which the pair rule accepts because its cable latency is positive:

When each rule is checked

Every rule above is checked when the upload completes. A file that breaks one is refused with 422 MAPPING_FILE_INVALID, the message names the first offending line (line <n>: <reason>) or the first missing rank, and nothing is stored. Two things depend on a configuration, so they are checked later:
  • Rank count. Attach requires the file’s rankCount to equal the rank count of the configuration’s traceset.
  • Chakra-capable servers. Whether each serverId names a Chakra-capable server is a fact about the topology. A file that names another server uploads and attaches, and the simulation fails when the simulator starts, with the simulator’s error naming the serverId.

Generate a mapping file

The platform does not generate a mapping file for a configuration. Scala provides a mapping file toolkit for writing one: a generator script, chakra_mapping_generator/generate_mapping.py, that writes a mapping file from a short JSON config; sample configs to start from, in chakra_mapping_generator/examples; and this documentation. Request the generator and its sample configs from Scala. The generator places rank i on serverId i, and can place the ranks in scale-up groups stamped from templates you define. Every file it writes follows the rules in The file format. It needs Python 3.10 or later and nothing outside the standard library.

Run the generator

Without --output, the file is written to standard output. When the config breaks a rule, the generator writes nothing, prints error: <field>: <reason>, and exits with status 2. The generator does not read your topology. It assumes servers 0 through rankCount - 1 are Chakra-capable servers in it. As with any mapping file, a serverId that is not Chakra-capable is reported when the simulation starts.

Start from an example

The chakra_mapping_generator/examples directory holds three configs, each with the file it generates beside it: Copy the one closest to what you need, set rankCount to your traceset’s rank count, and run the generator on your copy:
The bandwidth and latency values in the examples illustrate the format; they do not describe any particular hardware. Replace them with your fabric’s values. A rankCount that differs from the traceset’s rank count makes the attach fail with 422 MAPPING_RANK_COUNT_MISMATCH.

The config file

Template fields, in defaults or in an override: The three numeric fields accept the values Section II accepts (see Section II). Each template must also meet the bound on the two latencies as a pair, so a template with both latencies at 0 is refused. After overrides are applied, every template must have size, bandwidthGbps, perHopSwitchLatencyNs and npuCableLatencyNs. The generator has no built-in bandwidth or latency. Unknown and repeated keys are refused, so a misspelled field fails instead of being ignored.

How the config expands

  1. Build the templates. groupCount creates templates numbered 0 to groupCount - 1. Each starts as a copy of defaults, and its entry in overrides, if it has one, replaces the fields that entry names.
  2. Stamp groups in turn. Ranks are taken in order. Template 0 stamps the first group from the next size ranks, template 1 the next group from the ranks after that, and so on. After the last template the cycle returns to template 0. Every block is a new group, numbered 0, 1, 2, … in the order it is created, so group g uses template g mod groupCount.
  3. Number the subgroups. Inside each group, ranks are split in order into subgroups of subgroupSize. Subgroup ids restart at 0 in every group, and the last subgroup is smaller when subgroupSize does not divide the group.
  4. Stop at the last rank. When the ranks run out partway through a block, that last group has fewer ranks than its template’s size. It is still a group.
  5. Write the file. Section I has one row per rank: rank,serverId,ScaleUpGroupId,ScaleUpSubgroupId, with serverId equal to the rank. Section II has one row per group created, carrying its template’s bandwidth and latencies. Without templates, each row is rank,serverId and there is no Section II.
Groups stamped from the same template are separate scale-up domains, as two racks of the same model are. Traffic between them uses the scale-out network.

Worked example

The config above, with 18 ranks and two templates. Step 1, the templates. Template 0 has no override, so it is the defaults. Template 1 overrides three fields and inherits both latencies. Steps 2 to 4, the groups. Step 5, the file.
Section II has a row for each of the four groups, not for each of the two templates: groups 0 and 2 carry template 0’s values, groups 1 and 3 template 1’s. What the simulator does with it. Two ranks in the same group exchange traffic over the scale-up fabric at the group’s bandwidth; a rank sending to, or receiving from, several ranks at once shares that bandwidth equally among them. In the same subgroup the path crosses 1 switch hop and 2 cable segments; in different subgroups of the same group, 3 switch hops and 4 cable segments. Ranks in different groups use the scale-out network.

More cases

  • With rankCount 4 and the same scaleUp, only group 0 is created, with ranks 0 to 3: ranks 0 to 2 in subgroup 0 and rank 3 in subgroup 1. Section II has one row.
  • With rankCount 20, group 3 is full (ranks 16 to 19).
  • With no scaleUp, or groupCount 0, every row is rank,rank and the file has no Section II.
  • A whole number in a numeric field is written as a decimal: 800 is written 800.0, and 0 is written 0.0.
  • A file larger than 64,000,000 bytes, the upload limit, is refused before anything is written.

Upload, attach, and simulate

1. Set up your environment

See Authentication for how to obtain a token.

2. Request upload URLs

Send the file’s size in bytes and the workspace it belongs to. name is optional and defaults to chakra-mapping.txt. It must be 1 to 255 bytes of A-Z, a-z, 0-9, ., _ and -, must not be ., and must not contain .. anywhere. sizeBytes is an upper bound of at most 64,000,000 bytes (the 64 MB per-file limit), and description holds at most 2,048 characters. The name is display metadata only: inside the simulator the file is always called chakra-mapping.txt.

3. Upload the bytes

PUT each part to its presigned URL and keep the ETag response header of each one.

4. Complete the upload

Send the upload id and each part’s number and ETag. The platform validates the file, derives rankCount and scaleUpGroupCount, stores the file, and answers 201 with the new mapping file. sizeBytes in the response is the stored file’s actual length.
If the workspace already holds a live mapping file with the same bytes, under any name, the platform creates nothing and answers 200 with that existing file and duplicateOfExisting set to true. Uploading the bytes of a deleted file creates a new mapping file with a new id; a deleted file is never revived.

5. Attach it to a configuration

Attach requires, in this order of checks:
  • a configuration outside the template workspace (400 otherwise);
  • at least one application with trafficType chakra (422 MAPPING_REQUIRES_CHAKRA_APP);
  • an attached traceset (409 MAPPING_REQUIRES_TRACESET) that has a rank count: a traceset without one is answered 422 UNPROCESSABLE_ENTITY, attach changes nothing, and retrying does not change the answer;
  • a mapping file in the configuration’s own workspace that has not been deleted (404 otherwise);
  • a mapping file whose rankCount equals the traceset’s rank count (422 MAPPING_RANK_COUNT_MISMATCH, naming both counts and the missing or extra ranks).
If-Match is optional. The configuration’s ETag is its version number in quotes, and a stale one answers 412. Attaching writes a new configuration version whose activeMappingFile names the file (excerpt):
Attaching also sets UseMapping to true and MappingFileName to chakra-mapping.txt in every Chakra application’s ChakraConfiguration parameters. While the file is attached, the platform owns those two parameters: a configuration update, an application-parameter update, or a new Chakra application that would change them is refused with 422 MAPPING_PARAMS_LOCKED. A traceset with a different rank count is refused with 409 TRACESET_CONFLICTS_WITH_MAPPING. A configuration update whose body carries activeMappingFile is refused with 400: attach and detach are the only ways to set it. To make any of these changes, detach the mapping file, make the change, and attach it again. One mapping file can be attached to any number of configurations in its workspace, and each configuration holds its own reference.

6. Create a simulation

Every simulation created from the configuration is launched with the attached file (see Current limitations).
The simulation records the mapping file it was created with in appliedMappingFile from the moment it is created, and keeps naming it after the file is deleted, with deleted reporting that state. A simulation just created (excerpt):
A simulation created from a configuration with no mapping file carries no appliedMappingFile, and its ranks are placed in order. Creating a simulation is refused, and no simulation is created, when:
  • the attached mapping file has been deleted: 409 MAPPING_FILE_DELETED_REFERENCE, naming the file. Detach it, or attach another, and create again;
  • the attached mapping file’s name contains .. or a shell metacharacter: 400 VALIDATION_ERROR, naming the field. Upload rejects such names, but a file stored before that rule keeps its name; upload it again under another name and attach it in place of the old one.
A run never falls back to in-order placement: a mapping file that cannot be delivered to the simulator fails the simulation. When the platform rejects the file while staging it, for example because the stored object is missing, over the size limit, or fails its checksum, the simulation’s failureReason reads Mapping file <name> (<id>) could not be delivered. On some runs a long <name> is shortened and ends in …; the id is always given in full. A failure the platform does not attribute to the mapping file, such as a storage or infrastructure fault while staging it, reports a general failure reason instead.

7. Check the placement the run used

When a mapped run completes, the simulator has written the placement it applied to one CSV result file whose name contains -chakra-mapping-. List the run’s CSV results, find that file by name (page through with next if needed), and download it through its downloadUrl or the download-url endpoint:
The file’s header is ChakraId,Hostname,NicNodeID,ServerNodeID,ServerID,Core,IPAddress, plus ScaleUpGroupId,ScaleUpSubgroupId when the run uses scale-up groups. The first column is the rank and the fifth is the serverId it ran on, so each row can be compared with the Section I entry you uploaded. Rows are not in rank order. See Simulation Output Files for downloading result files.

Browse, download, and delete

List the live mapping files in a workspace, newest first. workspaceId is required, and limit is at least 1:
When hasMore is true, pass the returned nextCursor as the next query parameter to fetch the following page. Fetch one mapping file’s details, or a presigned URL (url, valid until expiresAt) that serves its exact uploaded bytes:
Delete a mapping file. The delete answers 204 whether or not a configuration refers to the file; deleting it again answers 404:
After a delete:
  • Simulations already created are unaffected. They keep the file they were created with, and their appliedMappingFile reads deleted: true.
  • A configuration that refers to the file keeps its activeMappingFile, which reads deleted: true, and creating a simulation from it is refused until you detach the file or attach another.
  • Fetching the file or its download URL answers 410 MAPPING_FILE_DELETED.
Detach a mapping file from a configuration to restore in-order placement. The mapping file itself is unchanged and can still be attached elsewhere:

Errors

All errors use the shared error shape:
See the Mapping Files API reference for every field and status code of each operation.