The Produce Market Guide (PMG) is a resource to produce commodity information, trends, and data analysis. PMG is a product of Farm Journal which shares information on fresh fruits and vegetables and is industry's leading source for news, information, and analysis.
The advanced search feature on the website was slow which ruined the overall user experience with increasing bounce rate. The previous version would take 17-25 seconds for a search to complete including the page load time and another 15-20 seconds for the results to populate.
Moreover, the product team wanted to monitor the user behavior on the search and provide them with the best personalized experience. The lack of any technological support and missing integration with search would cause the admin to only guess the user behavior pattern.
The organization was familiar with Drupal since the website was already on it sans the features they were looking for.
Since there were not many changes required in the architecture, the joint client-vendor team chose to progressively decouple the search functionality. The new search of the website was built with React & elastic whereas the overall architecture remained the same.
Homepage of PMG
About the project
The Goal of The Project Was:
- To build the search applications with facets and track user activities for a better and personalized user experience.
- Improve the user experience of the website by providing quicker/faster results
Requirements of The Project:
- Develop the architecture to index and map the data on the elastic server from Drupal to build an efficient modern search kit system
- Develop a backend service for the mapping and index the data on CRUD (Create, Read, Update, Delete) operations on the website
- Develop a Logstash pipeline to log queries (helps to track user behavior) on the elastic server and set up Kibana for visualization of data
- Write various test cases for the backend services and React app
- Improve the UI of the application as per the requirement
- Keep the launch time within the brackets of one month
A new design was planned to integrate the faceted search. It required four simultaneous implementations.
- Implementing ElasticSearch Connector module and Search API to create mapping and index on Elastic Server.
- Create a faceted search application with React using declarative components and integrate the app inside the Drupal as block or template page.
- Build custom features not available in the searchkit as per the client needs.
- Integrating Kibana and Logstash with ElasticSearch
Implementing the ElasticSearch Connector module
As per the requirement of the client a mapping was created on the ElasticSearch server and the data was indexed on the same. To meet the requirement of the indexing patterns used by Kibana and search kit, many custom preprocesses were written for the Search API.
Create a faceted search application with React
Once the Elastic backend architecture was developed, one of the biggest challenges in building the faceted search was adding the Searchkit and React in the same search application.
For that, the front-end, the logic and project structure for the search was designed and developed in the sandbox with modern tools like Babel and Webpack and 3rd-party libraries like Search-kit.
Faceted search result for a query
Why Search Kit?
The Searchkit is a suite of React components that communicate directly with the Elasticsearch cluster making the documentation of the user queries easier. It is fully customisable and therefore some of the custom features, which were not part of the search kit were also developed as part of the project.
UI of Searchkit
The following two features were part of the custom requirement:
- Multiselect Faceted query component
- Checkbox component
Integrating Kibana and Logstash
Both Logstash and Kibana are based on Elasticsearch. While Logstash is an open source tool for collecting and storing logs for future use, Kibana is a web interface which is a free open-source analytics and visualization platform to give a better understanding of the data.
Logstash and Kibana were then integrated on the elastic server to collect, parse, store, and visualize the data for better understanding. Both of these tools are based on Elasticsearch. Elasticsearch, Logstash, and Kibana, when used together is known as an ELK stack.
The app in the Sandbox was built for the production and all the CSS/JS was integrated inside the Drupal as a block to make it a progressively decoupled feature.
Back to topResults
The new PMG website is a lot user-friendly with search application and loads the results in lesser time than before. Since not much was added to the already existing website, the new version isn’t difficult to maintain.
Because of Logstash and Kibana, the admin can now track the user behavior on search application and personalize the user experience.
The project was completed within a month following the principles of SCRUM and AGILE with teams fluidly arranging themselves according to the needs.
Earlier the product would take 17-25 seconds for the search to load (including the page load time) and 15-20 seconds more after the page load for the results to populate.
OpenSense Labs helped PMG decrease their load time by 1200%.
The product now loads in less than 2 seconds and it takes less than 500 milliseconds for the results to populate.
Back to topWhy Drupal was chosen
- Customer’s prior experience with Drupal
- Flexibility
- Built-in API-first support
- Easy third-party integration
Technical Specifications
Drupal version:
Key modules/theme/distribution used:
ElasticSearch Connector Module - It is a set of modules designed to build a full Elasticsearch ecosystem in Drupal. It was used to create data mapping and index on Elastic Server. The module is used as a backend of Search API and all the CRUD operations were handled by the Search API itself.
