#set document(title: "3.4 Lifetimes, origins, and references", author: "Modular Inc. / XYZ Homework") #set page(width: 8.5in, height: auto, margin: 1in) #import "@preview/cetz:0.5.2" #set text(font: ("STIX Two Text", "Libertinus Serif", "New Computer Modern"), size: 10.5pt, lang: "en") #show math.equation: set text(font: ("STIX Two Math", "New Computer Modern Math")) #set par(justify: true, leading: 0.62em, spacing: 0.9em) #set enum(spacing: 1.1em) // room between list items so tall inline fractions don't collide #set list(spacing: 1.1em) #set table(stroke: 0.5pt + rgb("#c7ccd3")) #let BLUE = rgb("#183B6F") // brand navy — section bars + example/solution labels (white on navy 11.09:1) #let ORANGE = rgb("#A94509") // brand primary-700 — AA-safe deep orange for TEXT (5.93:1 on white; raw brand #F37021 is 2.94:1 and must never carry text) #let RED = rgb("#DC2626") // brand error-600 #let GREEN = rgb("#059669") // brand success-600 (decoration only; small green text uses green-text #007942) #show heading.where(level: 1): it => block(width: 100%, above: 0pt, below: 16pt, fill: gradient.linear(BLUE, rgb("#2C5AA0")), inset: (x: 14pt, y: 12pt), radius: 3pt, text(fill: white, weight: "bold", size: 19pt, it.body)) #show heading.where(level: 2): it => block(width: 100%, above: 18pt, below: 10pt, fill: BLUE, inset: (x: 10pt, y: 6pt), radius: 2pt, text(fill: white, weight: "bold", size: 12pt, it.body)) #show heading.where(level: 3): it => text(fill: ORANGE, weight: "bold", size: 12.5pt, it.body) #show heading.where(level: 4): it => text(fill: BLUE, weight: "bold", size: 10.5pt, it.body) #let examplebox(label, title, body) = block(width: 100%, breakable: true, fill: rgb("#EFF1F5"), stroke: 0.5pt + rgb("#CFDDF0"), radius: 4pt, inset: 10pt, above: 12pt, below: 12pt)[ #block(below: 6pt)[#box(fill: BLUE, inset: (x: 6pt, y: 2pt), radius: 2pt, text(fill: white, weight: "bold", size: 8.5pt, label)) #h(0.4em) #strong[#title]] #body] // rail = decorative left rule (raw brand token); labelcolor = AA-safe label text shade #let notebox(label, rail, labelcolor, tint, body) = block(width: 100%, breakable: true, fill: tint, stroke: (left: 3pt + rail), inset: (left: 10pt, rest: 8pt), radius: (right: 4pt), above: 11pt, below: 11pt)[ #text(fill: labelcolor, weight: "bold", size: 7.5pt, tracking: 0.5pt)[#upper(label)] #linebreak() #body] #let solutionbox(body) = block(above: 4pt, below: 8pt)[ #text(fill: BLUE, weight: "bold", size: 8.5pt)[Solution] #linebreak() #body] #let figph(msg) = block(width: 100%, height: 60pt, fill: rgb("#f6f7f9"), stroke: (paint: rgb("#c7ccd3"), dash: "dashed"), radius: 4pt, inset: 10pt)[ #align(center + horizon, text(fill: rgb("#889"), style: "italic", size: 9pt, msg))] // Standardize inlined figure sizes: measure the natural CeTZ canvas, then scale to a // consistent envelope (aspect-aware; see build_typst.py FIG_* constants). Unlike the // print preamble, dimensions are FLOORED: in an editor a user can trim a figure to a // degenerate 1-D shape (a bare line), and w/h or tw/w would then divide by zero. #let _STD_W = 3.5 #let _WIDE_W = 5.6 #let _MAX_H = 3.4 #let _ASPECT_WIDE = 2.2 #let _UPSCALE_MAX = 1.15 #let stdfig(body) = context { let m = measure(body) let w = calc.max(m.width / 1in, 0.01) let h = calc.max(m.height / 1in, 0.01) let tw = if w / h > _ASPECT_WIDE { _WIDE_W } else { _STD_W } let s = calc.min(tw / w, _MAX_H / h, _UPSCALE_MAX) align(center, box(scale(x: s * 100%, y: s * 100%, reflow: true, body))) } #show figure: set block(breakable: false) #set figure(gap: 8pt) #show figure.caption: set text(size: 8.5pt, fill: rgb("#555")) == 3.4#h(0.6em)Lifetimes, origins, and references The Mojo compiler includes a lifetime checker, a compiler pass that analyzes dataflow through your program. It identifies when variables are valid and inserts destructor calls when a variable's lifetime ends. The Mojo compiler uses a special value called an #emph[origin] to track the lifetime of variables and the validity of references. Specifically, an origin answers two questions: - What variable "owns" this value? - Can the value be mutated using this reference? For example, consider the following code: def print\_str(s: String): print(s) def main(): name: String = "Joan" print\_str(name) Joan The line name = "Joan" declares a variable with an identifier (name) and logical storage space for a String value. When you pass name into the print\_str() function, the function gets an immutable reference to the value. So both name and s refer to the same logical storage space, and have associated origin values that lets the Mojo compiler reason about them. Origin tracking and lifetime checking is done at compile time, so origins don't track the actual storage space allocated for the name variable, for example. Instead, origins track variables symbolically, so the compiler tracks that print\_str() is called with a value owned by name in the caller's scope. By tracking how owned data flows through the program, the compiler can identify the lifetimes of values. Most of the time, origins are handled automatically by the compiler. However, in some cases you'll need to interact with origins directly: - When working with references—specifically ref arguments and ref return values. - When working with types like #link("https://mojolang.org/docs/std/memory/pointer/Pointer/")[Pointer] or #link("https://mojolang.org/docs/std/memory/span/Span/")[Span] which are parameterized on the origin of the data they refer to. This section also covers ref arguments and ref return values, which let functions take arguments and provide return values as references with parametric origins. === Working with origins Mojo's origin values are mostly created by the compiler, so you can't just create your own origin value—you usually need to derive an origin from an existing value. Among other things, Mojo uses origins to extend the lifetimes of referenced values, so values aren't destroyed prematurely. ==== Origin types Mojo supplies a struct and a set of type aliases (comptime values) that you can use to specify origin types. As the names suggest, the ImmutOrigin and MutOrigin comptime values represent immutable and mutable origins, respectively: struct ImmutRef\[origin: ImmutOrigin\]: pass Or you can use the #link("https://mojolang.org/docs/std/builtin/type_aliases/Origin/")[Origin] struct to specify an origin with parametric mutability: struct ParametricRef\[ is\_mutable: Bool, //, origin: Origin\[mut=is\_mutable\] \]: pass Origin types carry the mutability of a reference as a boolean parameter value, indicating whether the origin is mutable, immutable, or even with mutability depending on a parameter specified by the enclosing API. The is\_mutable parameter here is an infer-only parameter. The origin value is often inferred, as well. For example, the following code creates a #link("https://mojolang.org/docs/std/memory/pointer/Pointer/")[Pointer] to an existing value, but doesn't need to specify an origin—the origin is inferred from the existing value. from std.memory import Pointer def use\_pointer(): a = 10 ptr = Pointer(to=a) ==== Origin sets An OriginSet is not a type of origin, it represents a group of origins. Origin sets are used for tracking the lifetimes of values captured in parametric closures. An OriginSet #strong[isn't] a general-purpose mechanism for expressing a combination of multiple origins. Instead, you can use origin\_of() to express an origin union. ==== Origin values Most origin values are created by the compiler. As a developer, there are a few ways to specify origin values: - Static origin. The StaticConstantOrigin comptime value represents immutable values that last for the duration of the program. String literal values have a StaticConstantOrigin. - Derived origin. The origin\_of() magic function returns the origin associated with the value (or values) passed in. - Inferred origin. You can use inferred parameters to capture the origin of a value passed in to a function. - Untracked origins. The untracked origins, MutUntrackedOrigin and ImmutUntrackedOrigin represent values that are not tracked by the lifetime checker, such as dynamically-allocated memory. - Wildcard origins. The ImmutUnsafeAnyOrigin and MutUnsafeAnyOrigin comptime values are special cases indicating a reference that might access any live value. Static origins You can use the static origin StaticConstantOrigin when you have a value that exists for the entire duration of the program. For example, the StringLiteral method #link("https://mojolang.org/docs/std/builtin/string_literal/StringLiteral/#as_string_slice")[as\_string\_slice()] returns a #link("https://mojolang.org/docs/std/collections/string/string_slice/StringSlice/")[StringSlice] pointing to the original string literal. String literals are static—they're allocated at compile time and never destroyed—so the slice is created with an immutable, static origin. Derived origins Use the origin\_of(value) operator to obtain a value's origin. An argument to origin\_of() can take an arbitrary expression that yields one of the following: - An origin value. - A value with a memory location. For example: origin\_of(self) origin\_of(x.y) origin\_of(foo()) The origin\_of() operator is analyzed statically at compile time; The expressions passed to origin\_of() are never evaluated. (For example, when the compiler analyzes origin\_of(foo()), it doesn't run the foo() function.) The following struct stores a string value using a #link("https://mojolang.org/docs/std/memory/owned_pointer/OwnedPointer/")[OwnedPointer]: a smart pointer that holds an owned value. The as\_ptr() method returns a Pointer to the stored string, using the same origin as the original OwnedPointer. from std.memory import OwnedPointer, Pointer struct BoxedString: var o\_ptr: OwnedPointer\[String\] def \_\_init\_\_(out self, value: String): self.o\_ptr = OwnedPointer(value) def as\_ptr(mut self) -\> Pointer\[String, origin\_of(self.o\_ptr)\]: return Pointer(to=self.o\_ptr\[\]) Note that the as\_ptr() method takes its self argument as mut self. If it used the default argument convention, it would be immutable, and the derived origin (origin\_of(self.o\_ptr)) would also be immutable. You can also pass multiple expressions to origin\_of() to express the union of two or more origins: origin\_of(a, b) Origin unions When a function returns a reference or pointer that can have one of several different origins, you can express the referenced origin as a union of all of the possible origin values. The union of two or more origins creates a new origin that references all of the original origins for the purposes of lifetime extension (so a union of the origins of a and b extends both lifetimes). An origin union is mutable if and only if all of its constituent origins are mutable. Use an origin union For an example, see Return values with union origins. Inferred origins Since origins are parameters, the compiler can #emph[infer] an origin value from the argument passed to a function or method, as described in Parameter inference. This allows a function to return a value that has the same origin as the argument passed to it. See the section on ref arguments for an example using an inferred origin. Untracked origins The untracked origins, MutUntrackedOrigin and ImmutUntrackedOrigin represent values that do not alias any existing value. That is, they point to memory that is not owned by any other variable, and are therefore not tracked by the lifetime checker. For example, the #link("https://mojolang.org/docs/std/memory/unsafe_pointer/alloc/")[alloc()] function returns an UnsafePointer to a new dynamically-allocated block of memory, with the origin MutUntrackedOrigin. The origin indicates that the memory is not managed by the Mojo ownership system. When you use an unsafe API like this, you're responsible for managing the lifetime yourself: for example, a struct that allocates memory should generally free that memory in its destructor. Wildcard origins The wildcard origins, ImmutUnsafeAnyOrigin and MutUnsafeAnyOrigin, are special cases indicating a reference that might access any live value. These were previously widely used for unsafe pointers. Using a pointer with a wildcard origin into a scope effectively disables Mojo's ASAP destruction for any values in that scope, as long as the pointer is live. It also prevents Mojo from enforcing argument exclusivity and hides unused variable warnings. Accordingly, the use of wildcard origins is discouraged, and should be used as a last resort. === Working with references You can use the ref keyword with arguments and return values to specify a reference with parametric mutability. That is, they can be either mutable or immutable. A ref return value looks like any other return value to the calling function, but it's a #emph[reference] to an existing value, not a copy. ==== ref arguments The ref argument convention lets you specify an argument of parametric mutability: that is, you don't need to know in advance whether the passed argument will be mutable or immutable. There are several reasons you might want to use a ref argument: - You want to accept an argument with parametric mutability. - You want to tie the lifetime of one argument to the lifetime of another argument. - When you want an argument that is guaranteed to be passed in memory: this can be important and useful for generic arguments that need an identity, irrespective of whether the concrete type is register passable. The syntax for a ref argument is: #strong[ref] #emph[arg\_name]: #emph[arg\_type] Or: #strong[ref\[]#emph[origin\_specifier(s)]#strong[\]] #emph[arg\_name]: #emph[arg\_type] In the first form, the origin and mutability of the ref argument is inferred from the value passed in. The second form includes an origin clause, consisting of one or more origin specifiers inside square brackets. An origin specifier can be either: - An origin value. - An arbitrary expression, which is treated as shorthand for origin\_of(expression). In other words, the following declarations are equivalent: ref\[origin\_of(self)\] ref\[self\] - An #link("https://mojolang.org/docs/std/memory/pointer/AddressSpace/")[AddressSpace] value. - An underscore character (\_) to indicate that the origin is #emph[unbound]. This is equivalent to omitting the origin specifier. def add\_ref(ref a: Int, b: Int) -\> Int: return a+b You can also name the origin explicitly. This is useful if you want to restrict the argument to either a ImmutOrigin or MutOrigin, or if you want to bind a function's return value to the origin of an argument. For example, the Span type is a non-owning view of contiguous data (like a substring of a string, or a subset of a list). Because it points to data that it doesn't own, it is parameterized on an origin value that represents the lifetime and ownership of the data it points to. In the following example, the to\_byte\_span() function takes a List\[Byte\] and returns a Span\[Byte\] with the same origin as the list: from std.collections import List from std.memory import Span def to\_byte\_span\[ is\_mutable: Bool, //, origin: Origin\[mut=is\_mutable\], \](ref\[origin\] list: List\[Byte\]) -\> Span\[Byte, origin\]: return Span(list) def main(): list: List\[Byte\] = \[77, 111, 106, 111\] \_ = to\_byte\_span(list) In this example, the origin parameter is inferred from the list argument, and then used as the origin for the returned Span. Since the Span takes on the origin of the list argument, the Mojo compiler can identify the span's data as owned by the list. The span will have the same lifetime as the list, and the span will be mutable if the list is mutable. ==== ref return values Like ref arguments, ref return values allow a function to return a mutable or immutable reference to a value. The syntax for a ref return value is: #strong[-\> ref\[]#emph[origin\_specifier(s)]#strong[\]] #emph[arg\_type] Note that you #strong[must] provide an origin specifier for a ref return value. The values allowed for origin specifiers are the same as the ones listed for ref arguments. ref return values can be an efficient way to handle updating items in a collection. The standard way to do this is by implementing the \_\_getitem\_\_() and \_\_setitem\_\_() dunder methods. These are invoked to read from and write to a subscripted item in a collection: value = list\[a\] list\[b\] += 10 With a ref argument, \_\_getitem\_\_() can return a mutable reference that can be modified directly. This has pros and cons compared to using a \_\_setitem\_\_() method: - The mutable reference is more efficient—a single update isn't broken up across two methods. However, the referenced value must be in memory. - A \_\_getitem\_\_()/\_\_setitem\_\_() pair allows for arbitrary code to be run when values are retrieved and set. For example, \_\_setitem\_\_() can validate or constrain input values. For example, in the following example, NameList has a \_\_getitem\_\_() method that returns a reference: struct NameList: var names: List\[String\] def \_\_init\_\_(out self, \*names: String): self.names = \[\] for name in names: self.names.append(name) def \_\_getitem\_\_(ref self, index: Int) raises -\> ref\[self.names\] String: if (index \>=0 and index \< len(self.names)): return self.names\[index\] else: raise Error("index out of bounds") def main() raises: list = NameList("Thor", "Athena", "Dana", "Vrinda") ref name = list\[2\] print(name) name += "?" print(list\[2\]) Dana Dana? Note the use of the ref name syntax to create a reference binding. If you assign a ref return value to a variable, the variable receives a #emph[copy] of the referenced item. Use a reference binding if you need to capture the reference for future use: var name\_copy = list\[2\] \# owned copy of list\[2\] ref name\_ref = list\[2\] \# reference to list\[2\] Parametric mutability of return values Another advantage of ref return arguments is the ability to support parametric mutability. For example, recall the signature of the \_\_getitem\_\_() method above: def \_\_getitem\_\_(ref self, index: Int) raises -\> ref\[self\] String: Since the origin of the return value is tied to the origin of self, the returned reference will be mutable if the method was called using a mutable reference. The method still works if you have an immutable reference to the NameList, but it returns an immutable reference: def pass\_immutable\_list(list: NameList) raises: print(list\[2\]) \# list\[2\] += "?" \# Error, this list is immutable def main() raises: list = NameList("Sophie", "Jack", "Diana") pass\_immutable\_list(list) Diana Without parametric mutability, you'd need to write two versions of \_\_getitem\_\_(), one that accepts an immutable self and another that accepts a mutable self. Return values with union origins A ref return value can include multiple values in its origin specifier, which yields the union of the origins. For example, the following pick\_one() function returns a reference to one of the two input strings, with an origin that's a union of both origins. def pick\_one(cond: Bool, ref a: String, ref b: String) -\> ref\[a, b\] String: return a if cond else b Because the compiler can't statically determine which branch will be picked, this function must use the union origin \[a, b\]. This ensures that the compiler extends the lifetime of #emph[both] values as long as the returned reference is live. The returned reference is mutable if #strong[both] a and b are mutable.