SphinxConnector.NET Features

SphinxConnector.NET is an all managed, fully asynchronous .NET client library to access the functions and features of the Sphinx full-text search engine and forks like Manticore Search. It offers developers several API choices for accessing Sphinx, the fluent API being the latest and greatest!

Fluent API

The fluent API provides you with a LINQ-like query API to design your full-text queries. It is tailored to the needs of .NET developers working with the Sphinx full-text search engine. Naturally, the fluent API supports tasks like ordering and grouping results, defining result-set sizes and projecting your results into e.g. anonymous types. In addition to querying, it also supports saving and deleting documents from real-time indexes for which it generates optimized statements depending on the context.

Under the hood, SphinxConnector.NET translates your input to SphinxQL, executes your query using its infrastructure for SphinxQL and maps the results to your document model. By using the SphinxQL API as the underlying query mechanism, users benefit from features like connection pooling, without having to manually write SphinxQL queries.

Please read on for a detailed description:

Querying Indexes

To start querying, you just have to create a class that represents your documents and you are good to go.

Example:

Index definition:
index books
{
	rt_field          = title
	rt_attr_string    = title
	rt_field          = author
	rt_attr_string    = author
	rt_attr_float     = price
	rt_attr_timestamp = releasedate
	rt_attr_uint      = ebookavailable
	rt_attr_multi     = categories
}
Document Model:
public class Book
{
   public int Id { get; set; }
   public string Title { get; set; }
   public string Author { get; set; }
   public decimal Price { get; set; }
   public bool EbookAvailable { get; set; }
   public DateTime ReleaseDate { get; set; }

   public IList<long> Categories { get;set; }
}

Querying Sphinx indexes is done via the IFulltextSession interface. It provides a method named Query that takes your document model as a generic type argument and returns an instance of IFulltextQuery.

IFulltextStore fulltextStore = new FulltextStore().Initialize();

using (IFulltextSession session = fulltextStore.StartSession())
{
    var results = await session.Query<Book>().
                                Match("@author Martin @* A Song of Ice and Fire").
                                Where(x => x.EbookAvailable).
                                OrderBy(x => x.Price).                                      
                                Select(x => new { x.Title, x.Price, ReleaseYear = x.ReleaseDate.Year }).
                                ToListAsync();                
}

The IFulltextQuery interface provides you with all the methods you need to easily order, group or filter your documents. SphinxConnector.NET automatically translates calls to supported methods and properties of .NET types to their equivalent SphinxQL functions. Supported are for example methods of the System.Math class and properties of System.DateTime. Additionally, SphinxConnector.NET defines extension methods like In and Interval for use in your queries.

Async methods

All fluent API methods have both synchronous and asynchronous implementations.

JSON Attributes

With the release of version 2.1.1, Sphinx introduced a new attribute type that allows users to store JSON objects in an index. SphinxConnector.NET supports these attributes since version 3.2 (via Json.NET at that time, since version 5.0 Json.NET has been replaced by System.Text.Json).

Future Queries

Future queries provide an easy way to optimize your performance in scenarios where multiple queries need to be executed. Future queries are only executed when their results are accessed. This allows SphinxConnector.NET to send multiple queries in a single network round-trip to the Sphinx server.

Future Query Example:
using (IFulltextSession fulltextSession = fulltextStore.StartSession())
{                
    var results = fulltextSession.Query<Product>().
                                  Match("a product").        
                                  ToFutureList();

    var facets = fulltextSession.Query<Product>().
                                 Match("a product").
                                 GroupBy(p => p.CategoryId).
                                 Select(p => new 
                                 { 
                                       p.CategoryId,
                                       Count = Projection.Count()
                                 }).
                                 ToFutureList();

   //No query has been executed up to this point

   foreach (res in results.Value) //accessing the result of any one query will cause both pending future queries 
   {                              //to be executed in a single round-trip to Sphinx at this point 
        //...
   }
}
Back to top

Saving & Deleting Documents

To save a document to a real-time index you only need to pass an instance of your document model to the Save method of the IFulltextSession interface:

using (IFulltextSession session = fulltextStore.StartSession())
{
    session.Save(new Book
    {
        Id = 1,
        Author = "George R.R. Martin",
        Title = "A Game of Thrones: A Song of Ice and Fire: Book One",
        EbookAvailable = true,
        Categories = new long[] { 1, 2 },
        Price = 5.60m,
        ReleaseDate = new DateTime(1997, 8, 4)
     });
     
     session.FlushChanges();
}

To delete one or more documents from a real-time index you can pass their id's to the Delete method or an instance of a document you would like to delete:

using (IFulltextSession session = fulltextStore.StartSession())
{
    session.Delete<Book>(1, 2, 3);
    
    session.FlushChanges();
}

As you can see from the examples, you call FlushChanges to let SphinxConnector.NET know that the changes you made should be made persistent.

Automatic Optimization and Batching

When you call FlushChanges, all pending saves and deletes are gathered and executed in the most efficient manner. For the example above, that means that all deletes are executed in a single DELETE statement thus avoiding unnecessary network round-trips.

When SphinxConnector.NET detects that it needs to save more than one document, it inserts them in batches by generating a single REPLACE statement for each batch. This leads to a speed-up by several orders of magnitude compared to inserting documents one by one. The batch size is of course configurable, so you can fine tune it to your workload.

Back to top

Convention over Configuration

SphinxConnector.NET uses a convention based approach for many of its configuration settings. For example, when it comes to mapping objects to indexes or properties to attributes, you do not have to provide any manual mappings or clutter your code with custom attributes. The mapping is done via conventions that are defined in the FulltextStore class. You can of course change the default conventions if necessary.

Example for an index name convention for a main-delta setup:
IFulltextStore fulltextStore = new FulltextStore().Initialize();
fulltextStore.Conventions.GetIndexName = (type, ctx) => String.Format("{0}, {0}_delta", type.Name.ToLowerInvariant() + "s");

Notice that the conventions are defined via delegates. Often you have to implement interfaces or inherit from base classes to change some behavior in a component that you are using. We think that most of the time, this poses an unnecessary overhead because the customizations are just a few lines of code. In these cases providing a simple delegate saves time, keeps the code clean and easy to maintain. And, in more complex scenarios, you have the option to implement abstractions the way you see fit without being limited or constrained in your design by some 3rd party interfaces.

Back to top

Object Mapping for SphinxQL

If needed, you can also execute SphinxQL queries directly and have SphinxConnector.NET map the results to your document objects:

Example

using (IFulltextSession fulltextSession = fulltextStore.StartSession())
{
    ISphinxQLExecutor sphinxQLExecutor = fulltextSession.Advanced.CreateSphinxQLExecutor();

    var parameters = new { query = "a product", categories = new[] { 23, 42 } };

    var results = sphinxQLExecutor.Query<Product>("SELECT * FROM products WHERE MATCH(@query) AND categories IN @categories",                                                   parameters);        
}
Back to top

Designed for Testability

All functionality of the fluent API is exposed via interfaces. By applying techniques such as Inversion of Control and Dependency Injection, you can easily test your code by substituting these interfaces by mocks if necessary.

Using the fluent API with your favorite DI container is also straightforward:

Example

Configuring Castle Windsor for SphinxConnector.NET
public class SphinxConnectorInstaller : IWindsorInstaller
{
    public void Install(IWindsorContainer container, IConfigurationStore store)
    {
        container.Register(Component.For<IFulltextStore>().Instance(new FulltextStore().Initialize()).
                                     LifestyleSingleton(),
                           Component.For<IFulltextSession>().
                                     UsingFactoryMethod(kernel => kernel.Resolve<IFulltextStore>().StartSession()).
                                     LifestylePerWebRequest());
    }
}
Back to top

Classic API's

SphinxConnector.NET provides high performance implementations of the native API and standard ADO.NET 2.0 classes for executing SphinxQL statements. Please see below for a list of features exposed by each API:

SphinxQL

  • Based on standard ADO.NET 2.0 classes:
    • SphinxQLConnection
    • SphinxQLConnectionStringBuilder
    • SphinxQLCommand
    • SphinxQLDataAdapter
    • SphinxQLDataReader
    • SphinxQLParameter
    • SphinxQLTransaction
  • Fully asynchronous
  • Tailored to Sphinx specific data types and features, e.g.
    • Automatic conversion from .NET types to Sphinx types and vice versa
    • Methods for handling MVA values
  • Insert and Update Records in Real-Time Indexes
  • Select from any Index Type
  • Select via DataReader or DataAdapter
  • Connection Pooling for Maximum Performance
  • Support for Transactions
  • Support for TransactionScope (Local and Distributed Transactions)
  • Support for Command Parameters
  • Automatically detects and uses features based on Sphinx version, e.g. server side multi-query support with Sphinx 2.0.1 and above
Back to top

Native API*

  • Support for Sphinx 2.0.1 and up
  • Support for string attributes introduced with Sphinx 1.10.1
  • Support for features introduced in 0.9.9 like:
    • Persistent connections: open one connection for several operations to minimize network overhead.
    • Override attributes: temporarily change the value of an attribute without modifying the actual value for advanced search scenarios.
    • Select clause: write SQL-like statements to operate on attributes
  • Configurable Encoding
  • Search related features:
    • Set value and range filters
    • Schedule several queries for batch execution
    • Specify how Sphinx should match and rank documents
    • and more
  • Access to additional functions exposed by the Sphinx search engine:
    • Build excerpts
    • Build keywords
    • Update attributes
    • Query Sphinx for status variables
  • SphinxConnector.NET uses custom types as arguments for methods operating on Sphinx attributes. This gives you strongly typed access to these methods and also enables the compiler to check whether the operation is valid for a given type.

*Please note that the native API is only intended to support legacy projects and will be removed in a future version. New projects should use the fluent API or SphinxQL.

Logging

SphinxConnector.NET supports logging different types of messages to help developers identify problems or optimize their setup. Since the release of V5 SphinxConnector.NET provides its own simple logging abstraction, replacing the previously used Common.Logging library. A simple Console logger is included in the main library, additional packages for NLog, log4net, Serilog, and Microsoft.Extensions.Logging are available via NuGet.

Supported Platforms

SphinxConnector.NET needs at least .NET 4.6.1 or .NET Core 3.1 to run and supports both Sphinx and Manticore Search (2.2.1 and up)..

In case you need to access a Sphinx version < 2.2.1 or cannot use .NET 4.5, you can still download SphinxConnector.NET V3 which runs under .NET 3.5 and supports Sphinx >= 2.0.1.