Flight Engineering: Aligning Multi-Channel Product Data
Wednesday July 19, 2017 | By Ethan Morgan
Flight Engineering is a series about developing and shaping the Volo platform to serve our customers' needs. This week, Solution Architect Alaric Snell-Pym explains how product data is handled differently on different marketplaces, and how we're solving the challenge of unifying product data for our customers.
As we discussed in a previous post, integration with multiple channels is dear to our hearts at Volo.
We've talked about how we deal with the software for talking to channel APIs, listing products and processing orders, but that's just one part of the process. As well as their own API, each channel has its own model for product data. For instance, products on Wish don't have categories; they just have a name, a description, a list of tags and some images; and optionally color, size and brand. Meanwhile, products on Walmart have a rich and complicated array of attributes. That includes a set of general attributes, a category, and then many more category-specific attributes.
What's the problem?
This causes a few complications. The fact that each channel expects this product data to be delivered in a different way (CSVs, XML, JSON documents or HTTP POST form parameters, to name the main categories) is the easy part; our pluggable channel architecture makes that the responsibility of the channel plugin, and the core system handles product data in the form of a JSON document, which it hands to the plugin to list.
But the keys in that JSON document are the attributes each channel expects. When the user wants to set up an import or export of their product data (or a regular feed), how can the system know what fields are available, to let the user choose which ones to map to which column? When the customer does an initial import before there are any products already known to the system, we don't even have any examples to work from.
One answer might be to make the channel plugins responsible for this - build the import/export/add/edit product data user interface inside them. If they provided this as a Web application, we could link the customer to it to deal with that channel's data for their products. But this would make it hard to provide a consistent and seamless user experience, and there would be a lot of duplicated work in the plugins to provide all the infrastructure of a Web-based product data user interface. This would make plugins harder to build and maintain!
Our solution to this problem is to make the channel plugins provide a "metadata document" about the channel. This is just a file that the Volo core gets from the plugin. It contains a description of the product categories and attributes supported by this channel; and it contains navigation information about what attributes are available in each category, how to group the attributes together in the user interface, help text, and validation information so we can help users to get their listings right first time. It also contains information about how things like variations work on the channel.
Making the user experience consistent
This means our product editing user interface, and our bulk import/export engine, can both handle product data for channels without needing us to write code for each channel. We can give users a beautiful, consistent, product editor, by using the metadata document to display the fields the channel needs.
Navigation through many hundreds of available fields is made easy: the navigation guides in the metadata let us present the user with an easily explorable interface, and the fact that we're generating that interface dynamically means we can offer features such as "Only show me fields whose names contain this word" or "Only show me compulsory fields" by just filtering the list of fields to display.
The metadata document isn't just for the product editor, of course. Most of the product data comes in via bulk imports, but the import tool uses the metadata document to let the user map from input columns to the correct fields in the product data JSON. The metadata contains everything that channel-agnostic code needs to handle channel-specific product data.
Each new channel plugin just needs to provide a metadata document, and be able to map the JSON product data into whatever format the channel needs it in; and then Volo Commerce can provide a beautiful and powerful browsing/editing/import/export interface without any additional work!