Prepend, Append... What?

In USD files you'll often come across terms like prepend and append whether you deliberately put them in there or not. If you look at a USD file in text form you'll see something like prepend references = @./filename.usd@. You may also notice that the file works as expected if you remove the prepend. So what's going on here?

Lists

There are many fields in USD that accept an editable list as the value. Some examples are variantSets, references, inherits, rel (relationships). If the list is defined on a single layer this is pretty simple. In text form, if it has one value, it's a list with one item. If it has a list in square brackets, it's a list containing those items.

Why prepend then? Read on my inquisitive friend!

Let's make up an example to demonstrate the idea. In this file I'm going to create some custom relationships that don't do anything. Each will have a list of other prims with names like "a", "b", "c". I'll write a little script to print out the composed value of the relationships, then we can look at different ways of building lists.

def Scope "Example"
{
    custom rel primList = <a>
    
    custom rel primListLonger = [<a>, <b>, <c>]

    def Scope "a" {}
    def Scope "b" {}
    def Scope "c" {}
    def Scope "d" {}
    def Scope "e" {}
    def Scope "f" {}
}

These rels are fairly useless, but you'll be able to author any other editable list in the same ways, so it should be good for a generic demonstration.

Here's a link to the script, but no need to understand it. It's only there to create consistent formatting for these examples. The relationships have these values.

primList         - ['a']
primListLonger   - ['a', 'b', 'c']

Combining Layers

So far so good. But what if we have two layers that set the value for the same list? The normal behavior in USD is that the strongest layer wins, and if we set the list value the way we did above that is what would happen. If we had a stronger layer in our stage and it had a value like the one below, that would replace the original definition.

custom rel primList = <b>

primList         - ['b']

Sometimes this is what you'll want. But imagine this was a list of variantSets, and what we really wanted was to add a new variantSet to the prim while keeping the variantSets that already existed in the list.

At authoring time we could get the list of variants that are already there and make a new list with the old and new values, [<a>, <b>]. That puts a lot of burden on the user to remember to do that, and it falls apart if the original prim's variants ever change. We'd need to go find every place we'd copied them into our layers and update them. If a third party publishes the asset it's even more complicated.

USD strives to make it possible for new content to be additive without requiring changes to the underlying data. This is key to the idea of "non-destructive editing". To avoid the maintenance nightmare of copying and editing lists, and to allow non-destructive edits to files, USD added this idea of editing lists across layers.

One layer can define a list, and stronger layers can make edits to the list. When USD reads the value from a stage it will resolve all the edits and create a final combined list value. This is the source of these prepend and append terms. Look at that; it only took me about 600 words to get there 😆

If we create these layers,

(
    subLayers = [
        @stronger-layer.usda@,
        @definitions.usda@
    ]
)

over "Example"
{
    rel primList = <b>
    prepend rel primListLonger = <d>
}

We can demonstrate both behaviors, one replaces the old list, and the other adds a new "d" to the front of the old list (in other words, prepending a "d").

primList         - ['b']
primListLonger   - ['d', 'a', 'b', 'c']

List Editing

There is some good documentation for this feature in the OpenUSD Glossary. List editing actually offers a few more operations than I've mentioned so far.

I'll make a quick little zoo where we can observe these keywords in action. The full files can be seen on github. I'll update definitions.usda to have a bunch of copies of the test relationships.

    custom rel primList_1 = <a>
    custom rel primList_2 = <a>
    ...

    custom rel primListLonger_1 = [<a>, <b>, <c>]
    custom rel primListLonger_2 = [<a>, <b>, <c>]
    ...

Then I'll add two stronger layers, both listed below. layer1.usda is weaker than layer2.usda.

over "Example"
{
    # Demonstrate each keyword that applies to single item lists
    append rel primList_1 = <b>
    prepend rel primList_2 = [<b>, <c>]
    delete rel primList_3 = <a>
    add rel primList_4 = <b>

    # Prepend once here, and again in a stronger layer
    prepend rel primList_5 = <b>

    # Demonstrate reversing the order of this list
    reorder rel primListLonger_1 = [<c>, <b>, <a>]

    # Here multiple operations are requested and they all apply
    prepend rel primListLonger_2 = <d>
    append rel primListLonger_2 = <e>
    delete rel primListLonger_2 = <b>
    
    # c is already in this list, rather than add it again this moves
    # it to the front
    prepend rel primListLonger_3 = <c>

    # This list has [a, b, c], reorder only b and c
    reorder rel primListLonger_4 = [<c>, <b>]

    # Pile on some operations, here and in the stronger layer, and
    # see what we get
    prepend rel primListLonger_5 = [<d>, <e>]
    add rel primListLonger_5 = <f>
    reorder rel primListLonger_5 = [<f>, <a>]
}

over "Example"
{
    # Prepend once here, and also in a weaker layer
    prepend rel primList_5 = <c>

    # Pile on some operations, here and in the weaker layer, and
    # see what we get
    delete rel primListLonger_5 = <e>
    reorder rel primListLonger_5 = [<c>, <b>]
}

On a stage we'll get these values.

primList_1       - ['a', 'b']
primList_2       - ['b', 'c', 'a']
primList_3       - []
primList_4       - ['a', 'b']
primList_5       - ['c', 'b', 'a']
primListLonger_1 - ['c', 'b', 'a']
primListLonger_2 - ['d', 'a', 'c', 'e']
primListLonger_3 - ['c', 'a', 'b']
primListLonger_4 - ['a', 'c', 'b']
primListLonger_5 - ['d', 'f', 'a', 'c', 'b']

Ok, Let's Answer the Question Already

I started out by saying that you'll see prepend references = @./filename.usd@ all the time even if you don't explicitly set it. Now we know what this does. If another layer also sets a reference on that same primitive, this won't delete it, instead it will prepend another reference. How did that prepend get into your file?

Lots of digital tools do this by default to try to be non-destructive. If they always prepend instead of setting the value they won't overwrite a list from a weaker layer. I'd say that 99.9% of the time this doesn't matter for references. It's pretty unusual in my experience to ever have multiple references on one prim.

If you ever do wind up with multiple layers defining a list of references on the same prim you'll see all the referenced items at once. If that's not your intention you can resolve this by making sure the references are on different prims, or by list editing the reference list to get what you intended.

Some Overly Detailed Trivia and Footnotes

• I originally wrote in this article that list edits are resolved during value resolution. While editing I realized this isn't the whole story. Since composition arcs can be list edited, those lists need to be resolved during composition. So list edits are resolved during composition and during value resolution.
• Time varying information can't be list edited. This means no USD Attributes are list editable, even though they can have list value types.
• add is interesting, since it seems the same as append. I suspect add may have existed before append and then was kept for backwards compatibility. I don't know this for sure though, don't quote me.
• reorder can get tricky if used too freely across layers. If all you care about is making one value strongest or weakest, it's a lot easier to reason about using prepend or append to move the value stronger or weaker.
• I thought this article would be so short it would hardly be worth writing. I am a fool. 🤡