App Sandboxing implementation

In the previous installment of Preparing for Mac App Store Submission, I discussed the reasons and methods for implementating Mac App Store receipt validation within your project.  At this point in the series, you may have already completed all of the currently necessary and recommended changes for converting your Mac OS X game, available for direct download, into an application suitable for the Mac App Store (MAS); you may have even submitted it already.

This final part is a discussion of the latest imposition by Apple, a technology known as application sandboxing, which was introduced with Mac OS X 10.7 Lion.  The general concept of sandboxing is that an application simply runs in its is own “sandbox”, unable to do anything outside its own little piece of the file system, except in specific cases, for which it must have entitlements.  The idea is that a compromised program (via malware or bad programming) is limited in the amount of system damage that can be done.  This is the way that all iOS applications work, and this is, in fact, just a transference of that approach to OS X.  The problem, of course, is that desktop systems are (and should be) different than tablets and phones, so certain types of programs are left out, including those which interact with other programs, such as those which provide ease of access to people who need assistance using computers.  (This is a very poorly considered decision on Apple’s part.)

Frankly, the main reason that you need to understand application sandboxing is that, as of June 1, 2012, all new applications and significant updates submitted to the Mac App Store are required to be sandboxed.  (This deadline was recently pushed back from the original date of March 1 in light of a considerable number of issues that Apple is still having with their implementation.)  If you already have a MAS application (or submit one before the deadline), you will be allowed to apply bug fixes without adding sandboxing, but otherwise, you will need to have sandboxing implemented.  (At least, that is what they are saying right now.)

0. Understand sandboxing

The first step is to understand application sandboxing and the implications of using it for your game.  The definitive documentation for Apple’s implementation of this technology is App Sandbox Design Guide, which is recommended reading.

Essentially, by default, your application is contained entirely within its own (read-only) bundle and a directory named by your bundle identifier (e.g., ‘com.goodsol.com/PrettyGoodSolitaire’) under the ‘~/Library/Containers’ folder.  An internal technology called Powerbox regulates (reads: mostly disallows) any access outside these two folders.  Most access to hardware features and user information is also disallowed without a specific entitlement.

If your game is entirely self-contained, then sandboxing may not present much of a problem.  If you use certain technologies, such as submitting high scores to a web site via the Internet, then you will have to enable appropriate entitlements.  If you have more involved interactions, especially involving other applications (or helpers), you may be forced to redesign/eliminate those features, or else give up on sandboxing (and, hence, the Mac App Store).  Whatever the situation, you will need to determine how this requirement will affect your application.

In our case, we lost the capability to load custom card sets, since the default operation (in the downloadable version) is to read installed files from the ‘Application Support’ folder, though that was not a huge blow since MAS already killed the idea of downloading improvements to the game.  The other feature that is gone is the ability to explicitly load and save games (for now) because the current version of our games are still built with Carbon, which cannot navigate the sandbox/Powerbox.  (Our primary save game feature, AutoSave, still works perfectly fine.)

1. Identify possible sandboxing issues

The next step is to identify possible issues with sandboxing your application, so let’s look at three different categories of features…

What you CAN do:

• You may read files from your application bundle, as usual.  (Bundles should always be considered read-only, so your code should never attempt to write here.)
• You may read and write files within your sandbox folders; this process is transparent provided you are using system calls to obtain the correct user folders (and you should never hard-code folder paths, anyway).
• You may update your application preferences (.plist) using a standard API.
• You may provide a simple link to a web page; this is allowed by default entitlements (i.e., not considered an outbound connection).

What you CANNOT do:

• You may not read from nor write to most locations outside your sandbox, including the user desktop and document folders, without the user explicitly selecting the location (and the appropriate entitlement set).
• You may not use any Carbon navigation dialogs, nor any Cocoa navigation dialogs other than NSOpenPanel and NSSavePanel, and you may not simulate user input to those panels.
• You may not use “incompatible” functions, including those included in the Authorization Services and Accessibility APIs.
• You may not send Apple events to arbitrary applications, nor broadcast user-info dictionaries.
• You may not load any kernel extensions.
• You may not alter preferences of other applications, nor terminate other applications, including helper applications.
• You may not sign future updates with a different certificate than the original; doing so will cause Mac OS X to deliberately crash the program.

What requires an entitlement:

• You need an entitlement to allow outbound network sockets (i.e., for sending information to the internet, operating as a client).
• You need an entitlement to allow inbound network connections (i.e., for acting as a server).
• You need an entitlement to allow read or write access to folders or files selected explicitly by the user (via NSOpenPanel/NSSavePanel).
• You need entitlements to access the user’s ‘Movies’, ‘Music’, and ‘Pictures’ folders.
• You need entitlements to access camera, microphone, or Bluetooth devices.
• You need entitlements to access serial, USB, or FireWire ports.
• You need an entitlement to allow printing.
• You need entitlements to access the user’s address book or calendars.
• You need an entitlement to allow the use of Core Location for determining geographical location.

After perusing the notes above, write down a list of features that may (or should) fall afoul of application sandboxing and be sure to test each of these features to confirm that they are working prior to enabling sandboxing entitlements.

2. Enable default entitlements

The next step is to enable the default entitlements for sandboxing; essentially, this disables all restricted access types, which allows us to verify which features of the game really need entitlements, and which work without modification.

To do this in Xcode 4.2.1 (under Lion) is fairly simple: Just open the ‘Summary’ tab for your target settings, scroll down to the ‘Entitlements’ section, and check the ‘Enable Entitlements’ (top) checkbox, and then confirm that the ‘Enable App Sandboxing’ checkbox is also selected.  If you are not already using iCloud in your project, then you should make sure that the ‘iCloud Key-Value Store’ is unchecked and cleared, and that that ‘iCloud Containers’ table is empty.

What the first checkbox actually does is create an entitlements file, a property list with the root name from ‘Entitlements File’ (defaults to the project name) and ‘.entitlements’ as the extension (e.g., ‘Pretty Good Solitaire.entitlements’), and it adds that file to your project.  The other (app sandboxing) checkbox automatically adds the ‘com.apple.security.app-sandbox‘ key to the entitlements file, which sets the default/maximum restrictions.

In order to build with application sandboxing enabled, you must be code signing the project.  Clearly this is already the case for projects being submitted to the Mac App Store.  However, if you are only testing and/or do not have an appropriate certificate (such as in our case where, as a contractor, I do not have access to the distribution certificate), you can follow the instructions in the ‘Create a Code Signing Certificate for Testing‘ section of the App Sandbox Quick Start.  [Editor: Sorry for no direct link, but section links on that page are fuzzled.]  The next section, ‘Specify the Code Signing Identity’, explains how to make the new certificate work with your project.

3. Verify access failures

Your next step is to verify access failures with application sandboxing enabled in the project.  To do this, start by launching the Console utility (from ‘Utilities’) and clearing its display.  This will show log entries to confirm a service denied by sandboxing.

Now, work through the list of suspect features that you created earlier and retest every one of those features with the new signed/sandboxed build.  Failures are, of course, expected, and for every failure there should be a log entry in Console from “sandboxd” and usually containing the text “deny” and an indication of the restricted service.

Note each confirmed failure, and determine which (if any) of the entitlements listed on the target ‘Summary’ page is likely to resolve the issue.  Do not make any changes to the entitlements until all of the suspect features are tested and notated.  A complete list of available entitlements can be found on the Enabling App Sandbox page of the Entitlement Key Reference.  (Note that some entitlements may not be listed on the ‘Summary’ page, requiring direct editing of the entitlements property file.)

If there are any features that are confirmed to fail only under sandboxing and for which there is not an appropriate entitlement, you may want to check the App Sandbox Temporary Exception Entitlements page for a solution that, as the name suggests, may work for the time being, but will probably not be available in the future.  If none of the entitlements nor temporary exceptions will resolve the problem, then you may need to redesign or remove that feature and/or lobby Apple to add an entitlement for the particular action you are attempting.

4. Add necessary entitlements

The next step is adding the necessary entitlements to the project, which should be done as an iterative process.  For each entitlement that you anticipate requiring, enable that entitlement, rebuild the project, then test every feature that is confirmed to have failed with sandboxing previously, noting each one that now functions correctly.

The general idea is that one should minimize the number of entitlements included (requested?) in a sandboxed application.  If you add an entitlement that does not actually help any of your features function, then you should immediately remove that entitlement.  Using excessive entitlements, in addition to reducing the (dubious) security, is a potential cause for rejection from the Mac App Store.  (In other words, simply checking all of the boxes may work in testing, but not for MAS submission.)

When you have completed this process, you should have a reasonably small number of entitlement keys (perhaps even zero) enabled, and all of the features of your program should be functioning properly.  It is always a good idea to double-check all of the identified features, as well as do some additional testing, once the “final” list of entitlements is enabled.  If you still have features that do not work under sandboxing (only), and have no appropriate entitlements, then you may have a decision to make.

In our case, we only had to add the ‘com.apple.security.network.client’ entitlement key, in order to perform submissions to our online high score server, but then we had to make the hard decision to lose the ‘Open…’ and ‘Save…’ features from our store version (for reasons stated previously).

5. Create file migration rules

The penultimate step in the process is creating file migration rules which is simply a property information file that is used only once (on a system) to indicate which existing files should be moved (not copied) into the sandbox.  If you have a new project, without existing users, then you have the luxury of skipping the final two steps.

The rest of us need to create a new property list, named (exactly) “container-migration.plist“, and add it to the project.  This file contains a dictionary key, ‘Move‘, which contains string keys indicating the names of folders and/or individual files to be moved into the equivalent location within the sandbox (or array pairs of strings if you wanted to both move files into the sandbox and relocate them within the relative folder hierarchy, though I cannot think of a good reason why you would).

The specific documentation for the (simple) format of this migration file is in Migrating an App to a Sandbox (which read).  The relevant portion of our migration file is as follows:

<dict>
    <key>Move</key>
    <array>
        <string>${Library}/Pretty Good Solitaire</string>
    </array>
</dict>

This is all it takes to move the entire (unsandboxed) ‘~/Library/Pretty Good Solitaire’ folder into the equivalent sandboxed folder (the unwieldy ‘~/Library/Containers/com.goodsol.PrettyGoodSolitaire/Data/Library/Pretty Good Solitaire’).

Important: Note that the files are moved to the sandbox; they are not copied (and there is no documented process to only copy them).  The ramifications of this are that existing users switching to a sandboxed version will automatically have their files and settings moved for them, but those files and settings will no longer be available to any unsandboxed versions, nor to any other application that may have been accessing these files.

6. Test file migration

The final step is to test the file migration process to assure that your migration file works correctly.

On a system on which you have unsandboxed data for the application in question, manually duplicate the folder and/or files (in ‘Finder’), giving you a backup for additional testing (especially important should the migration fail).  Confirm that the sandbox container folder (i.e., ‘~/Library/Containers/<bundle_id>’) does not exist (and delete it and empty the trash if it does).

Build the application with sandboxing enabled and with the ‘container-migration.plist’ file included in the project.  Run the application.  A container folder should have been created and the specified files/folders moved into appropriate locations within the container’s ‘Data’ folder.

If you need to retest (or if you decide against sandboxing entirely), you can completely delete the container folder for your application (‘<bundle_id>’ and below, not the ‘Containers’ folder) and copy the duplicated data back to its original location.

Conclusion

Following the above process to enable application sandboxing in your game, after having updated your project based on the previous five parts, you should be completely ready to submit to the Mac App Store.  Go for it!  Just keep in mind that the process changes, reviewers vary, and if your project is accepted on its first submission, you are very lucky.  However, if you use the available tools to check your code and your project build, and follow the guidelines in this series of posts, your path should be smoother.

I sincerely hope that you have enjoyed, or at least benefited from, all of these posts.  Comments are definitely appreciated, as are links to this blog, good word of mouth, and US currency in both large and small denominations.