How to Evaluate Thousands of Suppliers
The concept of Supplier Evaluation can be defined as the process of assessing and approving the potential line of suppliers of the company through various quantitative and qualitative evaluation measures. 
By examining a supplier’s capacities, technological resources, delivery strategies, and general business practices, it can be ensured that the supplier:
Shares the same priorities as the manufacturers
Has the ability to adjust their operations to the company’s competitive benefit.
Operates on low risks and has the capacity to manage any risk that arises
Believes in transparency and assists in improving visibility at every stage of the operation
Creates greater value for your products
Ensures quality of the raw material
Has the capability to cater to any fluctuations in demand that comes from the manufacturers
However, each manufacturing organization has a large number of suppliers and each supplier’s KPI would be different. For example, any supplier that deals with distributing magnetic material would have different flows of operations to ensure non- corrosive supply whereas a supplier that provides raw flour for edible items will have a very different working mechanism and protocols.
Usually the process of evaluating and scoring each supplier is an arduous and manual job. Additionally, its reporting is often done manually without any automated process resulting in a number of drawbacks such as:
Maintaining the evaluation system
Personal Bias of the evaluator
Identifying meaningful and relevant information from the evaluation metrics
Subjective or inconsistent storing
It is necessary to note that supplier evaluation is not only done during the scheduled supplier evaluations, but is also done at the time of deciding which suppliers to sign a contract with. It is necessary that a process is streamlined to automate the process of evaluating suppliers and categorizing their performance for easier reporting and assessment at scale.
Supplier evaluation is largely founded on data. Therefore, this should be a carefully structured process with foundations on different performance indicators such as delivery times, production costs, and inventory levels. A standardized set of supplier evaluation criteria provides a necessary frame of reference with which you can assess a supplier’s abilities and compare it with those of competitors.
The solution to determine the performance of suppliers and augment the process of evaluating suppliers is the following: it is necessary to first identify key metrics for the suppliers, set an infrastructure support for continuous data collection as suppliers get evaluated frequently within a fiscal year, determine the scoring mechanism of the suppliers to streamline evaluation, developing a mechanism to assess a new supplier against the existing supplier evaluation protocol, and finally to generate reports and alerts from the process.
Step 1: Identification of Evaluation Metrics
The key step in supplier evaluation is to determine the metrics or KPIs that the supplier will be evaluated against. These metrics can be generated by either a collective brainstorming exercise or using the existing evaluation metrics. Ng Shi Yun  suggests the following primary evaluation criteria for evaluating a supplier;
Figure 11: Key Supplier Evaluation Metrics
Source: https://sipmm.edu.sg /5-key-criteria-supplier-evaluation-construction-sector/
The most important factor of supplier selection should be the quality level of the procurement items. Product quality should consistently meet specified requirements since it can directly affect the quality of the finished goods. Supplier characteristics like delivery lead time should also be considered carefully. Additionally, the unit price should not be the only criteria in supplier evaluation but also the total cost of ownership should be considered. Total cost of ownership includes the unit price of the material, payment terms, cash discount, ordering and carrying costs, logistics and maintenance costs, and other more qualitative costs that may not be easy to assess. Suppliers must value add their product by providing good services when needed.
For example, when product information or warranty service is needed, suppliers must respond on a timely basis. Selecting services and products from suppliers with excellent delivery ability can reduce or get rid of waste related to purchasing raw materials such as inventory, storage costs, and costs related to multiple times of material transferring.
Ng Shi Yun  again describes the most important factors with their respective KPIs in the figure below;
Figure 12: Key Supplier Evaluation Metrics
Source: https://sipmm.edu.sg /5-key-criteria-supplier-evaluation-construction-sector/
Once the key metrics are gathered and KPIs are listed, the next step is handling the data collection of the highlighted KPIs.
Step 2: Streamlined Continuous Data Collection
When supplier evaluation is concerned, the data can come through various sources. It can either be from the report of a field visit by the assessment team or can come from interview/in-person meetings with the supplier. It could also come from general research about the supplier and critical analysis of their policies and working mechanism.
A big challenge before performing any scoring is to consolidate all these data points and store them digitally on a cloud for safe storage and accessibility. A very common procedure to record the findings is to usually develop a minimalistic mobile application that records all the necessary information. If such an application is not possible to be developed, a data entry operator would be required to input all this data to a digital platform.
Once the digital platform is ready, the process of saving information from supplier assessments/surveys is to be structured in such a way that it can be done whenever reassessments as well as evaluation of new suppliers is conducted.
Step 3: Supplier Categorization & Scoring
Upon getting the data in a structured format, it is important to note that before any methodology can be built, the priority of each KPI or Metric for each type of supplier needs to be defined. This is important as suppliers of different materials will have different working protocols and have some working procedures which are more important than others. For example, a supplier providing copper for wires would not be required to maintain a cool temperature in the room whereas a supplier of cocoa would be required to keep the cocoa beans at a certain temperature and humidity levels.
Once the priority of the evaluation metric is defined, a scoring mechanism needs to be built that utilizes priority in a quantifiable way to determine the performance of a supplier in the particular metric.
Another interesting outcome of the exercise could be the categorization of the suppliers based on their profiles/properties/metrics. This is known as ‘profiling’ in data science and helps in determining which ‘profiles’ are similar to one other against the different metrics that are defined. This also helps in identifying what key metrics would be important for a new supplier, which is further discussed in Step IV of this section.
The possible scoring mechanism in a non automated way can be as suggested in the book Purchasing & Supply Chain Management, by Monczka, Handfield, Giunipero and Patterson.
Step 4: Evaluating New Supplier
The question that arises at this point is that, ‘How can we evaluate new suppliers better?’. This is indeed a valid question because the way suppliers are evaluated currently does not include a process to help identify which metrics are important for the supplier. If we consider the usual scenario, it does not in any way utilize the existing suppliers’ behavior in understanding the priorities that are useful for the new supplier that needs to be evaluated.
A possible way to minimize the effort required in evaluating new suppliers is to perform clustering of the suppliers and their recorded properties. Clustering is a statistical process that groups together data that ‘resembles’ or is ‘similar’ to one other. When suppliers and their profiles get clustered together on the basis of their properties, new suppliers who do not have any evaluation data can be then grouped to the suppliers similar in nature and the new supplier can then be evaluated on similar metrics.
Step 5: Dashboard & Reporting
The most imperative part of any data science activity is to develop a dashboard that assists in understanding the results of the activity. An example of such a dashboard can be the following:
Figure 14: Sample Dashboard to Assess Suppliers