Eleventy - Merge data with an existing collection

26 May 2024

There are times when you would like to merge some data with an existing collection in eleventy. For example, I wanted to create an archive of my tech writing. I wanted to merge posts I wrote for other websites such as CSS Tricks with the posts I wrote on my personal blog. Everything in a single collection.

I couldn’t find an answer in the eleventy docs on how to include external data in an another collection. Here is how I did it.

TLDR

The simplest approach is to make a markdown file for each external post, the same as you do for your local posts, but provide the requisite data in the frontmatter.

My preference is to create a global data JSON file for the external posts. Having a clear separation should make it easier to manage.

Let’s name our file externalPosts.json and populate it with an array of posts. You can reference it in your eleventy configuration file by drilling down into the all collection as below:

module.exports = function (config) {
  config.addCollection("posts", (collection) => {
    let mergedPosts = [];
    let externalPosts = collection.getAll()[0].data.externalPosts;

    // get local posts and add to mergedPosts

    return mergedPosts;
  });

  // yada yada
};

There probably should be a function in the collection API to retrieve the collections created from data files directly!

My use case in full

I wanted to create a posts collection for all of my writing that was published on my blog and other websites. For example, I wrote 2 articles on CSS Tricks. There is no data source for that article metadata, so I need to manually create it.

Rather than make a markdown file for each external post, I would prefer to create a global data JSON file and merge it with the posts collection.

This is my project structure:

.
├── content
│   ├── blog.njk
│   └── posts
│       ├── 2024-01-01-post5.md
│       ├── post1.md
│       ├── post2.md
│       ├── post3.md
│       ├── post4
│       │   ├── possum.png
│       │   └── post4.md
│       └── posts.11tydata.js
├── _data
│   └── externalPosts.json
├── eleventy.config.js
├── _includes
│   ├── head.njk
│   └── layouts
│       ├── base.njk
│       └── post.njk

My “local” posts live in content/posts. I added the externalPosts.json file to hold the metadata for my external posts, they are available in page templates as an externalPosts variable. The problem is that you cannot concatenate this array with another array in a page template in a simple way. For example, Nunjucks does not have a merge filter or similar. You could create this filter, but I would prefer to do it in the eleventy configuration file in any case.

Here is an excerpt of externalPosts.json:

[
  {
    "url": "https://css-tricks.com/is-vendor-prefixing-dead/",
    "date": "2021-05-21T00:00:00Z",
    "data": {
      "title": "Is Vendor Prefixing Dead?",
      "description": "Browser vendors slowly began to move away from prefixing in 2012. It appeared that the problems created by vendor prefixes would fade away in time. The question is: has that time come yet?",
      "image": "/assets/img/external-posts/2021-05-21-prefixing-dead.webp",
      "publisher": "CSS Tricks",
      "tags": ["CSS"],
    }
  }
]

I structured it similar to the collection item object that Eleventy uses internally. This cuts out the need to transform the data later. The date field is a string because JSON does not have a Date data type.

In the eleventy configuration file, I did the following to merge this data with my “local” posts:

// eleventy.config.js

module.exports = function (config) {
  config.addCollection("posts", (collection) => {
    let localPosts = collection.getFilteredByGlob("content/posts/**/*.md");

    let externalPosts = transformExternalPosts(
      collection.getAll()[0].data.externalPosts
    );

    let allPosts = [...localPosts, ...externalPosts];

    // optional - I sort posts from most recent to oldest
    let sortedPosts = allPosts.sort(
      (a, b) => new Date(b.date).getTime() - new Date(a.date).getTime()
    );
    return sortedPosts;
  });

  // yada yada
};

function transformExternalPosts(posts) {
  return posts.map((post) => {
    let transformedPost = post;
    transformedPost.date = new Date(post.date);
    return transformedPost;
  });
}

You will notice that I have a transformExternalPosts function that converts the date field from a string to a Date object for the external posts.

Now, you can use the posts collection in your page templates. My blog page now includes the 2 posts from CSS Tricks.

Source code

The source code is available in this GitHub repo – https://github.com/robole/eleventy-tutorials. You can find it as an independent project in the external-posts folder.

Can you give the repo a star to indicate that this was a worthwhile tutorial please? 🌟

Tagged

• eleventy