Sitecore – Generating Item Resource Files

Update: this functionality is now available in the Sitecore CLI and in Sitecore TDS

With the release of Sitecore 10.1, Sitecore introduced the concept where Sitecore items are taken from the filesystem, instead of the database. A blog post by Martin Miles and a blog post by Jeremy Davis, explains this a bit more. Unfortunately, Sitecore does not give us an option (yet) to create those resource files ourselves.

As I’m not too fond on people telling me what I shouldn’t do, I started digging into the new resource file concept. It’s actually pretty easy, and it didn’t take me long to create a small console application that allows you to generate an item resource file, based on the item package generated by the Sitecore CLI. You could easily change it to any other input, as it just reads the serialized YML files to the protobuf structure needed by Sitecore.

Update:
Official support for the Item Resource files seems to be planned for the next version of the Sitecore CLI, and official documentation should be on the way as well.

Remarks

There are a few remarks that you should be aware of when you start using the item resource files:

  • The resource files are loaded alphabetically.
  • When an item or field exists in multiple resource files, the data is loaded from the last resource file where the item/field is found.
  • Make sure your resource files will be loaded after the default sitecore ones, so start your filename with a Z.
  • Make sure you place your resource file in the correct database folder, for the master database this is App_Data\items\master\.
  • This is NOT a support solution (yet) by Sitecore, so use it at your own risk.
  • Resource files are loaded once on the initialization of the application, so make sure you recycle your application to reflect your resource files.

Sample code

The sample code to generate a item resource file is displayed in the snippet below, read all the way down for a pre-made solution:

using System.IO;
using System.Linq;
using Microsoft.Extensions.Logging.Abstractions;
using ProtoBuf;
using Sitecore.Data.DataProviders.ReadOnly.Protobuf.Data;
using Sitecore.DevEx.Configuration;
using Sitecore.DevEx.Serialization.Client.Datasources.Filesystem.Configuration;
using Sitecore.DevEx.Serialization.Client.Datasources.Filesystem.Formatting.Yaml;
using Sitecore.DevEx.Serialization.Client.Datasources.Package;

namespace SitecoreResourceGenerator
{
    public class Example
    {

        public void GenerateProtobuf(string inputPath, string database, string outputPath)
        {
            ItemPackage package = new ItemPackage(new FileInfo(inputPath),
                PackageMode.Install,
                new FilesystemConfigurationManager(new NullLoggerFactory()),
                new FilesystemSerializationModuleConfigurationManager(new NullLoggerFactory()),
                new YamlSerializationFormatter(),
                new NullLoggerFactory());
            package.ReadModuleConfigurations();

            var serializationContainer = new ItemsData();
            foreach (var itemMetaData in package.ReadItemMetadatas(database).Result)
            {
                var item = package.ReadItemData(database, itemMetaData.Path).Result;
                serializationContainer.Definitions.Add(item.Id, new ItemRecord()
                {
                    ID = item.Id,
                    MasterID = item.BranchId,
                    Name = item.Name,
                    ParentID = item.ParentId,
                    TemplateID = item.TemplateId
                });

                if (item.SharedFields.Any())
                {
                    serializationContainer.SharedData.Add(item.Id,
                        item.SharedFields.ToDictionary(x => x.FieldId, x => x.Value));
                }

                var languageData = new ItemLanguagesData();
                foreach (var languageVersions in item.Versions.GroupBy(x => x.Language))
                {
                    var versionData = new VersionsData();
                    foreach (var version in languageVersions)
                    {
                        var fieldsData = new FieldsData();
                        foreach (var versionFields in version.Fields)
                        {
                            fieldsData.Add(versionFields.FieldId, versionFields.Value);
                        }
                        versionData.Add(version.VersionNumber, fieldsData);
                    }
                    languageData.Add(languageVersions.Key, versionData);
                }

                foreach (var languageVersion in item.UnversionedFields)
                {
                    var fieldsData = new FieldsData();
                    foreach (var versionFields in languageVersion.Fields)
                    {
                        fieldsData.Add(versionFields.FieldId, versionFields.Value);
                    }

                    if (!languageData.ContainsKey(languageVersion.Language))
                    {
                        var versionData = new VersionsData { { 0, fieldsData } };
                        languageData.Add(languageVersion.Language, versionData);
                    }
                    else
                    {
                        languageData[languageVersion.Language].Add(0, fieldsData);
                    }
                }
                serializationContainer.LanguageData.Add(item.Id, languageData);
            }


            var outputFilePath = $"{outputPath}.dat";
            using (var file = File.OpenWrite(outputFilePath))
            {
                Serializer.Serialize(file, serializationContainer);
            }
        }
    }
}

Pre-made solution

I’ve also created a Console application that is ready to use. As it requires Sitecore Libraries, I’m unable to supply you with a pre-built version, however you can check out the source code here, and build it yourself.

To use it, first create a Sitecore Serialization Package using the Sitecore CLI:
dotnet sitecore ser pkg create -o youroutputfile

Afterwards, run the Sitecore Resource Generator console application to generate the resource file:
SitecoreResourceGenerator.exe -i youroutputfile.scitempackage -d master -o z_customitems

This should generate the z_customitems.dat file, which you can then place in the App_Data\items\master folder.

Leave a Reply

Your email address will not be published. Required fields are marked *