Benchmarks
This page describes the input language, where the benchmark problems come from, and how to contribute a new domain. The output format your planner has to produce is described in the rules.
Input Language
All domains and problems are given in HDDL. What exactly that means for HTN IPC 2027 is defined by four things: HDDL itself, a document with clarifications, one addition (action costs), and a set of restrictions.
HDDL
Every domain and problem must be syntactically valid HDDL and follows the semantics of HDDL, as defined in the AAAI 2020 publication.
In particular, method preconditions have the default semantics of HDDL, as in IPC 2020 and IPC 2023: HDDL compiles them into an extra primitive task without effects, ordered before all other subtasks of the method. A method’s preconditions thus have to hold at some point after all actions that are ordered before the task it decomposes, and before the first action resulting from the method – not necessarily immediately before that action, as SHOP2 interprets them. If no action is ordered before the decomposed task, that point may also be the initial state. In the total-order setting, no action can be placed in between, so the two semantics coincide. The HDDL paper discusses both semantics, and why HDDL chose this one, in Section 3.2.
Clarifications
Some points of HDDL were not precise enough, so for IPC 2020 the organizers
wrote a document with clarifications.
IPC 2023 used it as well, and so do we. Among other things, it fixes which requirement
flags a domain declares, e.g., :equality whenever = is used, and that the
type object has no special meaning; in particular, it is not necessarily the
root of the type hierarchy. Note that we might update this document at some
point.
Addition: Action Costs
Action costs are new in 2027. They are used only in the action-cost optimal track, where every ground action has a fixed integer cost. In all other tracks, all actions have unit cost. The syntax is specified with the sample problems (to be released). The syntax will be the same as defined by PDDL; we will provide an exact pointer/explanation soon.
Restrictions
The competition uses only part of HDDL. As in IPC 2020 and IPC 2023, domains are restricted as follows:
- Preconditions, of actions and of methods, consist of literals (possibly
negative), conjunctions and universal quantification (
forall). - Effects consist of literals (possibly negative) and conjunctions only.
HDDL allows more, such as existential quantification, disjunction, universal quantification in effects, or conditional effects. None of these are used.
The Benchmark Set
The benchmark problems come from two sources:
- Domains from IPC 2020 and IPC 2023, with newly generated instances where a generator is available.
- New domains submitted in response to the call below.
Which domains and instances are used is not disclosed before the competition. Sample problems for testing are provided beforehand. The IPC 2020 webpage also links a range of example problems.
After the competition, all domains and instances are published, together with the submitted planners, their logs and plans.
Contributing a Domain
We call for new benchmark domains. Anyone may submit some, whether or not the competition is entered as well. We do not impose a limit on how many domains each person submits, but we make the final selection on which ones are used in the end.
A submission contains:
- the domain in HDDL, within the fragment described above;
- a generator for instances, accompanied by a set of instances of varying difficulty (though we might re-generate them);
- a short description of what the domain models and why it suits HTN planning. This is just for us internally, as we strongly encourage you to submit a short paper (just a few pages) that explains it in more detail. We will publish this as part of our non-archival IPC Booklet.
Ideally, your domain uses action costs: we can remove them with a script to obtain a version without them as well.
By submitting, you allow us to make the domain publicly available. We strongly encourage you to provide a permissive license, which makes it easier for others to change and extend it.
You may make your submission directly via email to the organizing team.
Last updated 6 October 2026.