Lua => lauxlib.c => luaL_loadfilex is an interesting case-study into how Lua passes information around internally. I became inspired to write this post when I stumbled upon this chunk near the beginning of the implementation ¹:

if (filename == NULL) {
  lua_pushliteral(L, "=stdin");
  lf.f = stdin;
}
else {
  // push
  lua_pushfstring(L, "@%s", filename);
  errno = 0;
  lf.f = fopen(filename, "r");
  if (lf.f == NULL) return errfile(L, "open", fnameindex);
}

Either =stdin or @<filename> are being pushed onto the Lua stack. But why? Where do these strings go? I inferred that this made it’s way into debug info about the source-code being loaded. There’s a clue in a distant section of Lua’s manual about The Debug Interface:

The fields of lua_Debug have the following meaning:

• source: the source of the chunk that created the function. If source starts with a ‘@’, it means that the function was defined in a file where the file name follows the ‘@’. If source starts with a ‘=’, the remainder of its contents describes the source in a user-dependent manner. Otherwise, the function was defined in a string where source is that string.

Part of this behavior, the prefix of = or @, is implemented in the string-pushing snippet early in luaL_loadfilex. The rest of this post details how it all comes together.

About the Stack

We’ve already seen that control flow starts with pushing =stdin or @<filename> onto the Lua stack.

Lua uses a virtual stack to pass values to and from C. Each element in this stack represents a Lua value (nil, number, string, etc.). Functions in the API can access this stack through the Lua state parameter that they receive.

Internally, Lua loves dogfooding its own stack. The internals are filled with what looks like a DSL operating on its own stack.

LUALIB_API int luaL_newmetatable (lua_State *L, const char *tname) {
  if (luaL_getmetatable(L, tname) != LUA_TNIL)  /* name already in use? */
    return 0;  /* leave previous value on top, but return 0 */
  lua_pop(L, 1);
  lua_createtable(L, 0, 2);  /* create metatable */
  lua_pushstring(L, tname);
  lua_setfield(L, -2, "__name");  /* metatable.__name = tname */
  lua_pushvalue(L, -1);
  lua_setfield(L, LUA_REGISTRYINDEX, tname);  /* registry.name = metatable */
  return 1;
}

Error Handling in Lua in C

One reason the stack may be used is because it plays nicely with Lua’s error handling behavior. According to Error Handling in C:

Internally, Lua uses the C longjmp facility to handle errors. (Lua will use exceptions if you compile it as C++; search for LUAI_THROW in the source code for details.) When Lua faces any error, such as a memory allocation error or a type error, it raises an error; that is, it does a long jump. A protected environment uses setjmp to set a recovery point; any error jumps to the most recent active recovery point.

Packaging values as Lua values and storing them on Lua’s stack helps ensure that values can be garbage-collected in case an error is encountered and this longjmp mechanism is invoked. In this example, the closed file handle is stored in a metatable on the stack first with this error-handling scheme in mind.

/*
** When creating file handles, always creates a 'closed' file handle
** before opening the actual file; so, if there is a memory error, the
** handle is in a consistent state.
*/
static LStream *newprefile (lua_State *L) {
  LStream *p = (LStream *)lua_newuserdatauv(L, sizeof(LStream), 0);
  p->closef = NULL;  /* mark file handle as 'closed' */
  luaL_setmetatable(L, LUA_FILEHANDLE);
  return p;
}

Convenience

The stack is necessary for causing little bits of Lua evaluation to happen inside a C program, which unsurprisingly happens often in the Lua implementation. It’s also important for error-handling and ensuring that values are available to be cleaned up by the garbage collector. The follow-on effect of the stack being a unifying principle in the Lua codebase is that utility functions will organize around references to values on Lua stacks. Inside luaL_loadfilex, the string =stdin or @<filename> is used several times.

It’s used on error-paths. errfile will get the string at fnameindex from the stack. It does this via access to all Lua state (”L”).

/* L822 */ if (lf.f == NULL) return errfile(L, "open", fnameindex);
/* L832 */ if (lf.f == NULL) return errfile(L, "reopen", fnameindex);
/* L844 */ return errfile(L, "read", fnameindex);

Here, it’s used as a checkpoint back to which the stack is unwound in case an error happens during lua_load.

/* L843 */ lua_settop(L, fnameindex);  /* ignore results from 'lua_load' */

Finally, on the happy path, the string is removed from the stack.

/* L846 */ lua_remove(L, fnameindex);

Tracing Further

We’ve covered what the Lua stack is all about and why this string is placed on the stack. Now, let’s push a few frames onto the good ’ol C stack and figure out how these magic strings eventually reach struct lua_Debug.

1. luaL_loadfilex calls lua_load. =stdin / @<filename is the third positional argument.
2. Inside lua_load, $3 is const char *chunkname.
3. chunkname is passed to luaD_protectedparser as $2
4. Inside luaD_protectedparser, $2 is const char *name
5. struct SParser p is allocated on the C stack (how gauche!)
6. f_parser, also defined in ldo.c is called by dynamic dispatch through a luaD_pcall. This was done in commit 13635f7de when I was 3 years old. I found this sort of intriguing because maybe the intent was to allow a user to possibly override the parser function. Actually, looking at the original diff, it seems to me more like this was done to dogfood the new implementation of protected calls at the time.
7. Inside f_parser, the opaque user-data pointer ud is a pointer to the same SParser p from before.
8. Later in f_parser, p->name is passed as $4 to luaY_parser.
9. Inside luaY_parser, $4 is name.
10. luaY_parser calls luaS_new, passing name as $2.
11. Inside luaS_new, $2 is str.
12. luaS_new means, “Lua string new.” It’s a string interning function.
13. Finally, the return value from luaS_new is stored in funcstate.f->source
14. Popping way back up to luaL_loadfilex, on line 846 the string previously pushed onto the Lua stack is removed now that it’s been copied into the closure.

This value lives with the closure for its lifetime and is read when we query debug info or when Lua prints a stack trace.