Helix Core / Perforce

Author

Flip Phillips

Note

This is more of an ‘explainer’. Go to the p4 and UE guide for the latest ‘howto’.

Helix (AKA Perforce)1 is a version control system that is used in many industries, most relevant here, the film and gaming industries.

Version Control (sometimes called Revision Control or Source Code Control) systems are used to track changes to files over time. They allow you to revert to previous versions of files, to see who made changes, and, as a result, effectively collaborate with others.

If you’re interested in the history of these systems, have a look at SCCS and RCS2. The most commonly used system currently is Git, which is free and open source. Git is excellent for software development, but it is not as well-suited for large binary files.3 This is where Perforce / Helix comes in.

Git is a distributed version control system, which means that every user has a complete copy of the repository on their local machine. This is in contrast to centralized version control systems, like Perforce, where there is a single repository that everyone connects to.

Distributed vs Centralized

In a distributed system all development can take place asynchronously. As each person’s changes are uploaded to the central repository (GitHub, GitLab, BitBucket, etc.), they are merged with the changes of others. This is powerful, but relies on the fact that changes to text files are relatively simple to merge. If I change line 30 and you change line 42 it is simply a matter of combining the two changes. If we both change line 44, then there is a conflict that needs to be resolved. With text files, these resolutions are relatively straightforward, but with binary files, like images, movies, or 3D models, the process is much more difficult. Changing a few pixels in an image can result in a completely different file, and merging two different images is not a simple process.

Perforce is a centralized system, which means that there is a single repository that everyone connects to. A rough metaphor for this is that of a lending library. When you want to work on a file, you check it out from the library. When you are done, you check it in. Two people can’t check out the same file at the same time (e.g., two people can’t change the same file at the same time). This means that there is a single source of truth, and that changes are locked until they are submitted to the repository. This eliminates the need for merging, but can lead to contention if two people want to change the same file at the same time. Even worse, the locking mechanisms can lead to situations where some files are locked that don’t need to be, or where files are locked and neglected. It requires good discipline to use a centralized system effectively, especially when there are many people working on the same files.

A Glossary

Here are some terms used in Perforce / Helix. Where I can, I mention some of the Git equivalents.

  • Depot
    • Centralized repository of ‘stuff’
    • In git it’s called a ‘repository’
    • Lives in the cloud / on a server at MAGIC in our case.
  • Workspace
    • Locally stored collection of files on the user’s workstation.
    • A copy of stuff from the Depot.
    • You can have stuff checked out on multiple workstations.
    • A ‘safe place’ to experiment with the project.
    • If you mess it up, you can ‘revert’.
  • sync / get latest / get revision
    • Sync downloads the latest version of the files.
  • change list / submit
    • A collection of changes made locally.
    • When work is done, work is submitted back to Depot.
  • add
    • add adds a file to the Depot.
    • Used for new files not already on the Depot.
  • metadata
    • Information about a digital asset and how it evolves.
  • revision
    • Tracks different versions of files in the Depot.
    • Most current version is called the ‘head revision’.
    • You can review and revert to older revisions.
  • checkout / open for edit
    • Before you change things, you must check them out.
  • locking files
    • You can place a lock on the file when you check things out.
    • Prevents edits by others.
    • Good for binary files - Keeps other people from editing it.
  • label
    • Can indicate a group of revisions that mark important collections of changes.
  • branch / streams
    • Copy of a project that can be worked on as a separate project.
    • main is central branch
    • You can create a branch to make major changes or dangerous stuff.
  • merge
    • Moving the files from the branch / stream back into the main stream / branch. Usually as a collection.
    • Reconciles differences.
  • resolve
    • When there are conflicts, you must resolve them.
    • You can choose to keep your changes, their changes, or a combination of both.

Footnotes

  1. Perforce is the company, Helix Core is the product. We’ll accidentally use them interchangeably.↩︎

  2. Your humble author started using SCCS as part of the Pixar rendering system he helped develop. Your humble author is old.↩︎

  3. Git-LFS ↩︎