Explaining drivers of service quality for future predictions

Discovery

Our discovery phase for this project involved looking into all the aspects of supply chain, starting from raw materials production to finish goods production, distribution centers composition and planned production pipelines.This involved collating information from various different sources and performing exploratory analysis to understand the dynamics of the supply chain & the factors influencing the actual fulfillment.

Through several meetings with the supply chain planners , we first tried to understand what are the key features that can influence the fulfillment rate.We developed a detailed feature engineering module that performed series of transformations on 100s of features available within the  database. This exercise provided us insights into building a plan to target the problem and move towards the solution step by step.

Proof of Concept

For our proof of concept we picked one category and approached the problem in two parts. 

Machine Learning Model to predict future dispatch rate

For each customer cross sub-category in product hierarchy we developed an algorithm that iteratively selects the best features to predict the fulfillment rate.The number of features extracted using this method was dependent on the number of available data points.So different sub category or product group had different number of features selected.Then within sub-category we ran a series of ensemble models to build a regressor that predicts fulfillment rate for next 4 , 8 & 12 weeks window. Here the best model was selected using a cross validation set up with hyper opt to extract the optimum hyper parameters from each model type. Our model had a Weighted Absolute Percentage Error of 5% on average, which was way lower than what they had before.

Attribution model to score drivers of prediction based on its influence on each prediction instance

Predicting the fulfillment rate was just a means to an end.The real objective of this project was to identify the levers the business can pull to mitigate the predicted loss from the model.To do this we first extracted all the lever variables for each product in each week over previous 26 weeks history. Then for each decline of the fulfillment rate we estimated the distribution of the lever variable. Now depending upon the decile of the predicted fulfillment rate where a particular prediction falls, we mapped the lever value from the features to the lever distribution obtained from history.Now if the likelihood of observing that lever value in the lever distribution corresponding to the fulfillment rate decile is more than 70% then we considered that lever to be significant for the prediction. For each product then we calculated this significance value which is defined as the probability of observing this particular lever value for future prediction in the historical distribution of lever value in the same decile of the fulfillment rate as the prediction.These scores provided supply chain planners to act on the significant lever to minimize the future loss.Examples of such levers are demand forecast accuracy , order lead time, production issue , safety stocks etc.

Scale

Our results were encouraging to the business, so the proof of concept was signed off. Then we further worked on this solution to generalize it so that this can be deployed to any standard data solution within our client's infrastructure. This proof of concept was done on one chosen region and category. Using this generalized automated solution we deployed the models in all categories for the chosen region and revisited the accuracies and the interpretability of the levers. The results on the scaled up deployments were consistent with our proof of concept and at this stage we had a fully automated solution to be scaled up at other regions as well.

Empower

Once we proved the value of our generalized automated modeling solution, it was the right time for us to share the deployment guides with our client's internal team to take hand over for further deployments. We created CI/CD pipelines and repositories for easy deployment and delivered all the documentations required. Finally we provided detailed training to the internal team for running the pipelines.

Support

Although the particular solution proved beneficial to the region where we did the proof of concept , however, for few particular region due to the nature of business it seemed that some more refinement was required. We provided support to our client on deploying this to other difficult regions by tuning the model and the features to target the uniqueness of that region. Once we tuned the model, we integrated the new model into the same global pipeline for continuous implementation.