Skip to main content

RFC-driven development

Status: Complete

Background​

RFC (Request for Comments) documents are well known as publications within communities used for proposing and iterating on technical standards.

The RFC system was invented by Steve Crocker in 1969 to help record unofficial notes on the development of ARPANET. RFCs have since become official documents of Internet specifications, communications protocols, procedures, and events.

The most famous of these formal RFC specifications define the protocols of the Internet itself, via IETF. Other projects have since adopted similar practices, such as Bitcoin with BIPs (Bitcoin Improvement Proposals) and EIPs (Ethereum Improvement Proposals).

However, RFCs were originally intended to be informational or experimental in nature, and not necessarily formal standards. That is, a way to write down notes and considerations around a problem, experiment with possible approaches, gather feedback, and share context with others.

Problem​

As a team of builders working towards shared goals, there's a need to iteratively explore problems and design solutions while gathering feedback and sharing context, ultimately arriving at a path forward for implementation.

This process should be collaborative and efficient, facilitate technical decision-making and design, with an emphasis on responsibility and trust within the team. It should be a lightweight process that facilitates iteration and forward movement, while gathering feedback, and sharing context. It should capture technical intent in a concise way, and avoid pitfalls like design by committee and bikeshedding

Similar lightweight processes for engineering teams have been described in the following:

Solution​

  • When you're faced with a problem that needs to be explored and/or decisions that need to be made, start by writing it down; the problem, goals, and any known constraints or requirements.
  • Begin your exploration process to understand and enumerate possible solutions or options. Break down the pros and cons of going with each, and any tradeoffs that would need to be made. You might even have to prototype pieces or all of the approaches to build confidence in and derisk them.
  • When you're ready for review and feedback, PR your RFC into the xmtp/eng repo under RFCs/XXX-name-of-your-rfc.md, tagging a couple reviewers. Seek feedback early, especially if you feel stuck or unsure of next steps. This stage should be iterative, with feedback and review resulting in updates, and discussion happening along the way. The purpose of this stage is also to share context and information with your team and stakeholders, not just to seek review from your selected reviewers; be open to review from others too. Just as with code changes via PRs, the expectation is that you include 1-2+ individuals for 👀 and 👍.
  • When you're ready, add a proposed solution with a basic implementation plan that can be translated into work items for our project boards, if you haven't already. The way you include your plan in the document is up to you, and can be as simple as bullet points, or more complex like a phased roadmap linked to Height tasks.
  • If questions are asked during review that would be useful for future RFC readers, then move those questions into the RFC under a Questions section with answers and reasoning, or wherever you deem appropriate in the document.
  • These documents are a snapshot-in-time, not living documents that are continuously updated long-term; there's no expectation that you continue to maintain the document after it's been implemented.
  • Your document can use this template if you want, some variation of it, or something completely different, it's your call.
  • A document status should be included in the header of the document and in the index entry, and should be draft, review, complete, or abandoned. You are responsible for updating the document status throughout its lifecycle.
  • We lean heavily on responsibility and trust within our team, and so it's your call as to when you want to mark the RFC as complete or abandoned, but the expectation is that you have clear and objective reasoning that aligns with our goals, and that you communicate and solicit feedback along the way. Consensus among everybody is absolutely required, but it is important that you maintain trust of your peers and stakeholders.

Plan​

  • Open a PR to xmtp/eng with this RFC, under RFCs/001-rfc-driven-development.md and add it to the the RFC index in RFCs/README.md.
  • Merge the PR when ready.

Questions​

  • Are these documents intended to be our public specifications of the XMTP protocol?
    • No, these are internal documents intended to facilitate decision-making, context sharing, forward movement, and capture technical intent. The formal specifications should be captured and published separately, in documents like BIPs, EIPs, Waku2 RFCs, or Solid RFCs, perhaps called XIPs (XMTP Improvement Proposals).
  • How long should these documents be?
    • Write as much or as little as you need to. Ideally make it concise and avoid unnecessary verbosity. The goal is to get the right information across and move forward on the work. Sometimes this needs a page or two, sometimes it needs 10, it's your call.
  • Where should I write these documents?
    • Write it wherever you're most comfortable, and shop it around to others in that form if you want. When you're ready, PR it into this repo as markdown, and tag your reviewers on the PR.
  • How should I share these documents?
    • Open a PR into this repo, and make sure you tag reviewers as PR reviewers.
  • Am I responsible for doing all of the work described in the implementation plan section of my RFC?
    • The work doesn't need to be all on you, just the responsibility to break down the problem and a path forward in the document. Often, you will be the most familiar with the work and will want to push forward with implementation, but sometimes you'll want to pull in other engineers to divide and conquer it. It's your call, assuming you can convince others that it makes sense to do.
  • What are the criteria for what should or shouldn't be approached with an RFC like this?
    • For example, I might want to note somewhere the decision to choose a tool like pnpm over npm and lerna, but I don't necessarily want to write up a detailed RFC.
      • It's your call as the engineer responsible for it. If it helps you think through a problem or communicate some context, even if small, then great, somebody will be happy to review and help you think through that. If you don't feel the need for one then great too; your PR description would probably communicate the same context in most of those cases.
    • Does a broad and/or vague problem like "SDKs" fit as a RFC?
      • It's your call again as the engineer responsible for it. When approaching a big vague problem like "SDKs" it can be useful to break it down in a RFC-like structure. These RFCs tend to sketch out the structure or architecture of a bigger system, and break down the exploration work into sub-components of it that can then spin off as their own targeted RFCs. These overarching RFCs are important because they can set the stage for the big areas of work ahead for a team.