author: zep // picotron.net
updated: 2023-11-08 (WIP!)
This is a technical design document for Picotron's filesystem -- not everything here will initially be implemented, and might change.
Picotron has its own virtual filesystem that is roughly mapped onto the host operating system's filesystem, with a few additional features to support Picotron's workflows and security model:
- .p64 cartridge files are logically folders inside Picotron
- files / folders can be mounted to a temporary drive in memory (/ram)
- processes can each have their own limited / transformed view of the filesystem
- files are atomic objects than correspond to Lua objects
The filesystem design follows from the goal of supporting cartridge-orientated data storage and development; there is a single global Working_Cartridge in memory that can be found in /ram/cart. Various tools can operate on files inside that cartridge, but it is written as a single .p64 file on the host machine in a compact and shareable format.
Parts of the host filesystem can be accessed by Mounting them anywhere inside Picotron's virtual drive, and processes can each have their own limited view of the filesystem using Sandboxing. This protects against malicious userland programs accessing the host file system, and also against malicious bbs cartridges messing with the virtual picotron drive.
Each file stores a single POD (Picotron Object Data), which is an unstructured tree of data that corresponds to a Lua object. For example, a table of sprites can be directly saved to disk with: store("foo.pod", my_table_of_sprites).
To keep this format interoperable with typical host machines, a text (or binary) file is considered to map to a single Lua string, and can be edited with a regular text editor if desired.
For more information on PODs, and fetch()/store():
https://www.lexaloffle.com/dl/docs/picotron_pod.html
Each file and folder also has a separate metadata fork in POD format. It can contain arbitrary data written by any process with sufficient permissions. It is typically used to store things like "author" and "version". There are some fields that are automatically written by store():
created date the file was created (YYYY-MM-DD HH:MM:SS)
modified date the file was last modified (including modifying metadata or copying the file)
revision an integer that is incremented by 1 each time the file is saved
On the host machine, metadata is stored inside each file itself, or for folders in a hidden file called ".info.pod". This way, files and folders can be moved with being separated from their metadata. When metadata is not present, some values (created, modified) are inferred from the host operating system's attributes where possible.
Apart from metadata, each file has 3 special attributes that can be read with fstat(path):
flags 0x1 directory // is set when path is logically a folder (including .p64s)
origin when the path is a mount point, this describes the mount point's origin
size size on disk in bytes
Cartridge files (.p64, .p64.png) in Picotron are logically folders. Even though they appear as files on the host filesystem, they can be treated inside Picotron the same as any other folder. For example, to copy some graphics from one cartridge to another, standard file utilities can be used:
cp cart1.p64/gfx/0.gfx cart2.p64/gfx
The contents of cartridges files can be exposed as regular files on host though, e.g. to edit them with tools outside of Picotron. see: Mounting_Cartridges
An application in Picotron is generally a single cartridge that contains everything it needs to run. So there is no need for a package manager or installation process; applications are installed by copying the cartridge to a preferred location. Data written by the application is stored separately in /appdata, so no special process is needed for updating to a new application version.
In general, Picotron treats every file as a single POD in a consistent way. There are two exceptions to improve interoperability with host files / editors:
1. .png files are treated as a 2d userdata [with palette metadata?]
2. text files (.txt, .lua) are stored almost as-is on host. A metadata attribute is stored on the first line (pod_format="raw"), but a new text or lua file is loaded from host with no metadata, it is also assumed to be in that raw format.
Picotron is a single-user machine, and aims to have as few special locations as possible so that the user can decide where to store their data, including a mess of cartridges in / if desired. There are 5 folders that exist on a clean install and have special meanings, and can not be moved:
/system
/ram
/desktop
/apps
/appdata
A read-only directory containing everything Picotron needs to function, including built-in libraries, resources and bundled tools.
A virtual ram drive that disappears when Picotron is shutdown. It is sometimes used for processes to communicate with each other, and to cache the contents of .p64 folders. A temporary file/folder can be created anywhere on disk by mounting it to /ram.
/desktop is the default location for workspaces running in desktop mode. There is another desktop folder in /appdata/system/tooltray/desktop (the hidden fold-out desktop thing)
The apps folder serves mostly as a way to organise various types of applications stored on the machine, to create a unified user-facing directory. They might link to pre-installed tools in system, or to bbs carts installed via splore (the built-in online cartridge collection browser), or be the application itself.
This folder is used by applications to store data, including system settings and save games.
A location is a string that identifies a particular file (the file name) and optionally a location inside that file. For example, to refer to foo.lua line 32, the following location string is used:
foo.lua:32
Locations can use either relative or absolute paths. The portion after the string can be arbitrary and the meaning is up to the program opening it.
Location files normally have the entension .loc, and contain a field: "location", and optionally some environment variables for opening that location. They can also have their own metadata and icon. This makes it possible to run programs or open files in a particular context with their own icon.
There are no symbolic links in Picotron; instead a file can be mounted somewhere else for that session using mount() (or the mount command).
Host paths can be mounted in only two ways: via a configuration file read at boot (picotron_config.txt), or by dragging and dropping a file into Picotron from the host operating system. This is to create a safe boundary between Picotron and the Host's filesystem; in each case it is assumed there is a clear intention to access that portion of the host operating system.
picotron_config.txt can contain mount commands, where the second parameter for each one is a host path:
mount / ~/picotron_drive
When no mount points exist, / is mounted in ram.
When dropping a host file into the (host) picotron window, the file / folder is mounted in /ram/mount and the window inside picotron that is active will receive a "dropped_file" message.
To mount a path inside picotron's filesystem to another point inside picotron's filesystem:
mount /myproj/temp /ram/foo
The contents of a .p64 files are automatically mounted in /ram/mount and periodically flushed to disk when changes have been made. This normally only happens during development; sandboxed cartridges can not write to themselves, and even local cartridges typically store their data in /appdata.
The working cartridge can be mounted using a regular folder on host as the origin, in order to use external tools:
mkdir /live_cart
mount /ram/cart /live_cart
folder /live_cart
Now, pressing CTRL-R will essentially look for main.lua in /live_cart, and operations like saving and loading the cartridge will copy to and from that folder.
Processes have roughly 3 levels of access to disk:
kernel: full access, including mount paths from host (process id 1~3)
userland: can access only picotron's filesystem
sandboxed: can access only a subset of picotron filesystem
A sandboxed process is one that has its own limited view of picotron's filesystem. For example, a cartridge running from splore can only see the following folders:
/system -- (read-only)
/cart -- (read-only) self; the contents of the .p64
/ram/shared -- (read-only) read-only system state
/appdata -- mounted at parent's /appdata/bbs/{cart_id}
The program file / cartridge itself is not tagged with permissions (like in UNIX); it is the process launching the program that ultimately decides what permissions it should have. However, .p8.png cartridges are always considered to be untrusted and are run sandboxed.
The third parameter of create_process() can be used to specify a custom mount list, relative to the process creating it. Access permissions of a child process can at most be a subset of permissions of the parent process:
create_process("/foo.p64", my_env, {{"/desktop", "/desktop", "rw"}, {...}})
Read-only access to /system, /cart and /ram/shared is always granted and does not need to specified.
Cartridges downloaded and launched by splore are given their own storage area, mapped to /appdata.
From inside the child process, it is possible to e.g. store a save game with:
store("/appdata/savegame3.pod", gamedat)
This is mapped to:
/appdata/bbs/cart_id/savegame3.pod
When running a cartridge locally (with userland permissions), /appdata is not mounted at a special location, and so all of the local programs use the same folder. It is considered courteous to keep data in a sub-folder named after the app (or author) if the cartridge is likely to be run in that context:
store("/appdata/wobblepaint/pic3.wob", wobdat)
Sandboxed programs can have additional mount points given to them during runtime by userland programs. For example, when a file open request is made from a sandboxed program, filenav.p64 (userland) can mount the selected file inside the requesting processes's filesystem so that it can be read and/or written. The same applies to dragging and dropping a file into a sandboxed program; it is taken to be a signal that read/write access to that single file or folder is intended by the end user.
These locations are normally not used in distributed programs, but are useful during development or for writing specialised tools.
/ram/cart
This is the location of the cartridge currently loaded into memory (there is only every one). The load command simply copies it from the source location, and CTRL-S / save command copies it back to that location. The filename of the currently loaded cartridge can be found in /ram/system/pwc.pod (present working cartridge).
/ram/compost
Compost is a place to move files that should be eventually deleted, but if you make a mistake and fish them back out quickly they are probably still ok.
/appdata/system/startup.lua
This script is run on boot if it exists. Handy for opening up a preferred set of tools and workspaces, or setting up a custom working cartridge mount point.
/ram/shared/theme.pod -- current desktop theme data; use theme() to access this
/ram/shared/user.pod -- (logged in user info -- for future use)
