For the penultimate door of the Drupal Advent Calendar, we are joined by Mike Herchel (mherchel) to tell the amazing story of bringing Single Directory Components into Drupal Core.
How we brought SDC into Drupal Core (and how you can too!)
Single Directory Components (SDC) started out as a rough idea in early 2022, had its first contrib release in March 2022, was proposed to go into core in October 2022, and was committed to core just 6(ish) months later! In the general schedule of Drupal core development, this was a light speed!
In this story I’ll walk through how the idea came about, and how we worked with the community to get this into core… and more importantly, how you can reproduce our success!
Step 1: Get your idea
The first step in submitting something into core is figuring out what problem you’re going to solve. A good rule of thumb is that if there are multiple solutions for the same issue (like how paragraphs, microcontent, storage entities all do the same thing), it’s likely a prime candidate for core inclusion.
If you want to have your idea vetted before working on it, you can submit it to the Drupal Ideas issue queue. This is a project on Drupal.org where anyone can submit ideas. It doesn’t mean anyone will work on it, but it will get core committers looking at the idea, and can give you some feeling on if it has hope to get into core.
SDC’s roots first started in early 2022, when Mateu Aguiló Bosch was working for a large client at Lullabot, and saw a need for a better way to manage front-end components.
At the time, the client was using Fractal to manage these and bringing them into Drupal using some custom code. There were some difficulties, especially with Twig.js, which Fractal uses to render the components. Twig.js is a JavaScript implementation of Twig, but it has some weirdness around the way it renders. In addition, the web development industry was standardizing around Storybook, which is a competitor to Fractal.
In this case, Mateu came up with the idea from seeing an obvious need that his client faced. He knew there were some solutions in Drupal, but nothing that worked quite the way that was needed.
Step 2: Assemble your team
For a decently large piece of functionality to go into core, you’re going to need several roles (of which one person might fill several), which may include
- Backend developer - The person who will be responsible for the majority of the PHP, config, database integration etc. Ideally this person has experience with core development.
- Code reviewer - These people will be responsible for reviewing code in a timely manner, so changes can quickly be made. Ideally these developers have worked on similar issues in the past and are also familiar with core development.
- Someone that can write tests - This can be the backend developer, but also someone else. This person needs to be able to write unit and functional tests. Functionality that is not thoroughly tested will not make it into core.
- Frontend developer - This person can write custom CSS, JavaScript, and ensure that any UI additions meet Drupal’s high standards of accessibility.
- Project manager - This person will make sure that issues are being worked on, and are categorized properly. They’ll also bring in new people to the project as necessary and solicit reviews. They’ll also work on keeping any issue summaries up to date (this is important!)
- Hype person - Another (less appreciated) role is someone to write about the project and give the community updates. This is crucial, because it 1) keeps the community (including core committers) informed of your status, 2) brings people into the project, 3) Puts pressure on people (including core committers) to provide timely reviews. This person will ideally blog about the project often, post on social media, and talk about it at conferences.
- Core committer - This is someone on the core team that can serve to answer quick questions, give a bit of mentorship as necessary, and overall guide to core.
For SDC, Mateu was the obvious backend developer. He wrote the module, was enthusiastic about it, and also had experience getting functionality into Drupal core (being one of the driving forces behind core’s JSON:API integration). Mateu also wrote the tests, as he was familiar with the process.
For reviewers, we were lucky to get a number of amazing people including (but not limited to) Pierre Dureau (a maintainer of UI Patterns), and Lee Rowlands (a core committer).
I (Mike Herchel) served as the frontend developer, project manager, and hype person 🙌. I was the person who suggested to Mateu that we try to bring this into core. I also wrote frequent blog posts, social media posts, and generally did my best to keep up everyone’s enthusiasm!
Lauri Eskola served as our primary contact within the core committer team. We frequently talked with him about our various API ideas, and he would poke holes in them and make us think things through.
Outside of the people mentioned above, there were many more who helped out with API design, gave support by giving us thumbs-up’s, left additional code reviews, etc. Thank you all!
Step 3: Write code in contrib
Typically you’ll need to vet your idea by creating a contrib module/theme if it doesn’t yet exist. This will give people a chance to download it, use it, file bug reports, request features etc. This also gives core committers an idea how people will use this new functionality.
With SDC, Mateu came up with the contrib module before considering trying to get it into Drupal core. The need for this was obvious, as there were a number of different solutions to this problem, but none that the community coalesced on.
The initial implementation of SDC was the Component Libraries module, which Mateu wrote about in June ‘22. It was a departure from the various ways that people have been managing components in Drupal (including UI Patterns, Compony, Emulsify, etc). It used Drupal to actually render the data in the component management system, and did a great job at organizing components.
Step 4: Create an issue (with a merge request) to add it to core
Assuming that your contrib code went well, and you have core committer buy-in (from your Drupal Ideas issue or another way), it’s time to create your issue.
This issue is vitally important because this might be the first time some people have heard of this. You’ll need to make sure that the issue summary is filled with good information and well thought out. This step will get more community acceptance, and bring more to the table.
Single Directory Component’s issue can be found at https://www.drupal.org/project/drupal/issues/3340712. In it, you’ll see several sections, all which were consistently kept up to date by Mateu and I. These include
- Problem / motivation
- Proposed resolution
- Remaining tasks
- Validation proving community buy in
- Frequently asked questions
- UI changes
- API changes
- Data model changes
Once you create this issue, let people know! Write blog posts, tell people about it on social media, and on Slack! You’ll want to ask people to comment on it to show community buy in, and click that “Follow” button.
Step 5: Hurry up and wait (and then hurry)!
Once you get your MR in a good place and you have it set to “Needs Review”, you’ll obviously need people to review it. You’ll want people to
- Manually test (and post screenshots)
- Look at the code (and leave reviews)
You’ll inevitably get feedback on both and assuming the feedback is valid, you’ll want to [very] quickly respond in the issue queue thanking the reviewers! Then you’ll want to fix the issues as soon as you can, and ask them for re-reviews.
Eventually, you’ll get to the point where it looks good, and someone will put it into RTCB (reviewed and tested by the community) status… but don’t rejoice just yet…
The next reviewer will likely be a core committer and they can bring in the hurt (in a good way)! They’ll ruthlessly review your code line by line and make sure that it makes sense, has good practices, and is efficient and well maintainable. Once again, you’ll need to quickly respond and thank them for the review (remember, many are volunteers). Then you’ll need to resolve the issues, and rinse/wash/repeat as much as needed.
Once the first core committer gets through it, it’ll likely go to another (the final boss!). This person, once again will find issues (a fresh set of eyes…), and you’ll need to thank, fix, and repeat, several times. Sometimes you’ll get questioned on the overall architecture… and you’d better be able to defend it. In our case, we had to qualify how single directory components made sense versus single file components.
If you’ve gotten your project this far, it’s likely that it’ll eventually make it in. Just keep it up and don’t get discouraged. You’ve got this!
Step 6: Move out of alpha
When new modules and themes get first committed to Drupal core, they go into an experimental state. Basically, this means that they’re in core, but they may have API changes, and they also may be removed if they don’t work out.
When your experimental project first gets into core, it goes in as an alpha release. This means that APIs are definitely not stable. It also means that this won’t be in the actual consumer-level Drupal release (via Composer). For people to use it, they need to pull down Drupal core via Git.
The next step past alpha is beta. Beta experimental modules are considered API- and feature-complete for the minimum viable product. To move to beta, you’ll need to create another issue. You’ll need to use this as a meta issue to track all beta-blocking issues identified.
In SDC’s case, we considered our code beta-worth as soon as it was in core. That being said, we still had to open an issue to make our point and get core committer buy-in, write documentation, etc.
Step 7: Stabilize!!!
The next step after beta is stable. This is when it formally becomes part of core and is as integrated as it can get.
To get to this point, you’ll need to open up yet another issue to track all stable-blocking issues. Some of these could be as simple as letting time pass for people to find bugs and provide feedback.
At this point it helps to continue to blog, and speak about the new functionality as much as possible. Hopefully, you’re presenting on this at DrupalCons, DrupalCamps, and everything else in between! Doing so gets more people using the feature and leads to more feedback (which is a good thing!).
Single Directory Components is not yet stable, but you can check out our roadmap issue. As with all issues in this process, it’s vital to keep it up to date with the current progress. Our hope is that we stabilize in Drupal 10.3, which is due in June 2024!
Conclusion
Hopefully these steps can serve as a guidepost and motivation to get your own ideas into core! It takes a long time, and can be very discouraging at times. But other times, it can be exciting and overwhelming. I get excited thinking about how people will be using SDC in the future, and how it will help people learn to love Drupal’s front-end all over again. Happy coding and happy holidays!
Mike Herchel is a senior front-end developer at Agileana currently working on big federal Drupal projects. In addition to helping out with Single Directory Components, he was the driving force and developer behind Drupal’s new default theme, Olivero. He also worked with other friends to get features such as Views Responsive Grid, and Twig Debugging UI into core. He’s passionate about all things open source and open web, including web accessibility, and performance. You can frequently find him at various Drupal events (including the best Drupal camp ever), and lounging in his hammock with his two dogs and one 13 year old Swiftie daughter.
Comments