Better frontend search

[discobot]

For years and years (frontend) search has been a complete disaster. There are several plugins that have tried to improve the search functionality, some with better results than others, but should something as fundamental as search be done with a plugin? I think not.

---

Read-only archive: Issues · ClassicPress/ClassicPress · GitHub

Author: Pieter Bos

Vote count: 34

Status: open

Tags:

• difficulty-hard
• request-modify-feature

---

Comments

Cool idea but require a feature plugin and an imp0lementation to see what we can do as core.

~ posted by Daniele Scasciafratte

This is a huge project, and most of the pros use Elasticsearch to improve this.

Another widely-used option is https://searchwp.com/.

~ posted by James Nylen

I think search is plugin territory, similar to how a contact form is plugin territory.

~ posted by John

I think search is a core function for a CMS, so I agree with this idea in principle.

But, as @James Nylen says, this is a huge project.

~ posted by Tim Kaye

Thanks for your support, Tim.
Maybe something to put on the list for ClassicPress v3.0.0

~ posted by Pieter Bos

Pieter, that would certainly be something to aim for!

~ posted by Tim Kaye

I’d like to see ‘relevance’ reflected in search results more than ‘publish date.’ I don’t think plugins are the ideal way to handle this fundamental feature.

~ posted by Ray Gulick

Relevanssi is the other option: https://www.relevanssi.com/

~ posted by Fabian Wolf

WP core has a number of queries that scale extremely poorly with large numbers of posts, comments, users, etc. The default search falls under that category too.

~ posted by James Nylen

Default search only makes sense for blogs. If we’re serious about being primarily for business websites, we need to change that (not necessarily to an ultimate search package). Plugins would still be useful to extend the non-bloggy basic search.

~ posted by Ray Gulick

the fundamental problem with search is the underlying MySQL database, which as was pointed out above, most who have a critical need for search, will reach for a scalable solution such as elasticSearch or a SAAS offering like Algolia. SearchWP as a plugin made some improvements to the front end, but ultimately you’ll be left with a pig wearing lipstick as long as the search relies on MySQL, and you have a non trivial dataset.

For reference:
https://hackernoon.com/dont-waste-your-time-with-mysql-full-text-search-61f644a54dfa

From MySQL full-text search to Elasticsearch | Scout APM Blog

Leveraging Elasticsearch for a 1,000% performance boost | Elastic Blog

~ posted by Paul van Zyl

I use ElasticPress. But would prefer that a better search should be native. After all, search is what made the internet what it is today.

~ posted by Anonymous

I would strongly recommend starting this feature as a plugin, that way we can iterate there and recommend the plugin for testing.

If you’d like to put something up on GitHub under the ClassicPress-research name, let me know and I’ll create a repository for it. ClassicPress-research/better-search is one idea for the name.

In order to be considered for merge into ClassicPress core, the plugin should have excellent automated test coverage and consistent code style.

~ posted by James Nylen

A better front-end search, in my opinion, wouldn’t be about providing all possible features within mysql (which is obviously going to fall flat), but about providing a search framework that can provide a better experience out of box, and which makes it easier to extend with additional functionality in plugins.

The core features I would like to see in an improved search are:

1. a filter_cache table with columns for object_id, ‘object_type’, filter_type, filter_val, and filter_text. filter_var, search_type, and object_type would be 191 character varchars with a b-tree key on it, and filter_text would be a text field with a fulltext index on it.
2. a register_filter_cache() function that defines specific data (e.g. post_content, specific meta_fields, term names) that should automatically be copied into the filter cache and kept up to date on change, as well as registering any transformation functions that should be run on the data before storing and whether the data should be stored in the value field or the text field. Also might allow you to create named search queries for easy reference in WP Query later.
3. a filter_query parameter for the various query objects that allows you to define which filters to include or exclude and how to prioritize them.
4. the “s=” query parameter would be retained as a shortcut to a specific default filter_query.

This system would make the default search experience much more effective, by allowing filter functions to strip tags and shortcodes, or remove specific shortcodes whose content isn’t meant to be searchable (e.g. private content shortcodes), and would also be able to power more complex filtering functionality for plugins and themes, which is currently completely custom and very inefficient without relying on custom tables.

Lastly, it would give us a better framework to use when dropping in external search solutions, such as elastic search.

~ posted by Greg Schoppe

Search is complex. One can search for content type A on page 1, content type B on page 2, or both content types A and B on page 3. One way this can be done, I think, is to enhance the search widget functionality by default. Like, when creating a search widget, there is an option to search including custom fields, option to search based on CPT, and option to filter using custom fields/taxonomy based on the CPT chosen.

~ posted by Raymund

Please create an option to completely disable front-end search.
While search IS crucial for many websites, including non-bloggy business ones, there are also lots of business sites are supposed to be completely private and for those front-end search is just an extra headache.

~ posted by ALS

@ALS: please start a separate petition for disabling front-end search.

~ posted by James Nylen

[viktor]

Improving front-end search is a big topic. So I wanted to bring attention to Greg’s suggestions that should be reviewed and implemented if feasible.

@discobot

A better front-end search, in my opinion, wouldn’t be about providing all possible features within mysql (which is obviously going to fall flat), but about providing a search framework that can provide a better experience out of box, and which makes it easier to extend with additional functionality in plugins.

The core features I would like to see in an improved search are:

1. a filter_cache table with columns for object_id, ‘object_type’, filter_type, filter_val, and filter_text. filter_var, search_type, and object_type would be 191 character varchars with a b-tree key on it, and filter_text would be a text field with a fulltext index on it.
2. a register_filter_cache() function that defines specific data (e.g. post_content, specific meta_fields, term names) that should automatically be copied into the filter cache and kept up to date on change, as well as registering any transformation functions that should be run on the data before storing and whether the data should be stored in the value field or the text field. Also might allow you to create named search queries for easy reference in WP Query later.
3. a filter_query parameter for the various query objects that allows you to define which filters to include or exclude and how to prioritize them.
4. the “s=” query parameter would be retained as a shortcut to a specific default filter_query.

This system would make the default search experience much more effective, by allowing filter functions to strip tags and shortcodes, or remove specific shortcodes whose content isn’t meant to be searchable (e.g. private content shortcodes), and would also be able to power more complex filtering functionality for plugins and themes, which is currently completely custom and very inefficient without relying on custom tables.

Lastly, it would give us a better framework to use when dropping in external search solutions, such as elastic search.

~ posted by Greg Schoppe

[namithj]

Close (don’t need it, close it)

This is better done as a plugin. The search requirement vary and a filter would be good but the use cases are too varied to include this in core.