SphinxConnector.NET Features

SphinxConnector.NET is an all managed .NET client library to access the functions and features of the Sphinx full-text search engine from any .NET application. It offers developers several API choices for accessing Sphinx, the fluent API being the latest and greatest!

Fluent API

The fluent API provides you with a LINQ-like query API to design your full-text queries. It is tailored to the needs of .NET developers working with the Sphinx full-text search engine. Naturally, the fluent API supports tasks like ordering and grouping results, defining result-set sizes and projecting your results into e.g. anonymous types. In addition to querying, it also supports saving and deleting documents from real-time indexes for which it generates optimized statements depending on the context.

Under the hood, SphinxConnector.NET translates your input to SphinxQL, executes your query using its infrastructure for SphinxQL and maps the results to your document model. By using the SphinxQL API as the underlying query mechanism, users benefit from features like connection pooling, without having to manually write SphinxQL queries.

Please read on for a detailed description:

Querying Indexes

To start querying, you just have to create a class that represents your documents and you are good to go.

Example:

Index definition:
index books
{
	rt_field          = title
	rt_attr_string    = title
	rt_field          = author
	rt_attr_string    = author
	rt_attr_float     = price
	rt_attr_timestamp = releasedate
	rt_attr_uint      = ebookavailable
	rt_attr_multi     = categories
}
Document Model:
public class Book
{
   public int Id { get; set; }
   public string Title { get; set; }
   public string Author { get; set; }
   public decimal Price { get; set; }
   public bool EbookAvailable { get; set; }
   public DateTime ReleaseDate { get; set; }

   public IList<long> Categories { get;set; }
}

Querying Sphinx indexes is done via the IFulltextSession interface. It provides a method named Query that takes your document model as a generic type argument and returns an instance of IFulltextQuery.

IFulltextStore fulltextStore = new FulltextStore().Initialize();

using (IFulltextSession session = fulltextStore.StartSession())
{
    var results = session.Query<Book>().
                          Match("@author Martin @* A Song of Ice and Fire").
                          Where(x => x.EbookAvailable).
                          OrderBy(x => x.Price).                                      
                          Select(x => new { x.Title, x.Price, ReleaseYear = x.ReleaseDate.Year }).
                          ToList();                
}

The IFulltextQuery interface provides you with all the methods you need to easily order, group or filter your documents. SphinxConnector.NET automatically translates calls to supported methods and properties of .NET types to their equivalent SphinxQL functions. Supported are for example methods of the System.Math class and properties of System.DateTime. Additionally, SphinxConnector.NET defines extension methods like In and Interval for use in your queries.

JSON Attributes

With the release of version 2.1.1, Sphinx introduced a new attribute type that allows users to store JSON objects in an index. As of version 3.2, SphinxConnector.NET supports these attributes by leveraging the well-known JSON.NET library.

Future Queries

Future queries provide an easy way to optimize your performance in scenarios where multiple queries need to be executed. Future queries are only executed when their results are accessed. This allows SphinxConnector.NET to send multiple queries in a single network round-trip to the Sphinx server.

Future Query Example:
using (IFulltextSession fulltextSession = fulltextStore.StartSession())
{                
    var results = fulltextSession.Query<Product>().
                                  Match("a product").        
                                  ToFutureList();

    var facets = fulltextSession.Query<Product>().
                                 Match("a product").
                                 GroupBy(p => p.CategoryId).
                                 Select(p => new 
                                 { 
                                       p.CategoryId,
                                       Count = Projection.Count()
                                 }).
                                 ToFutureList();

   //No query has been executed up to this point

   foreach (res in results.Value) //accessing the result of any one query will cause both pending future queries 
   {                              //to be executed in a single round-trip to Sphinx at this point 
        //...
   }
}
Back to top

Saving & Deleting Documents

To save a document to a real-time index you only need to pass an instance of your document model to the Save method of the IFulltextSession interface:

using (IFulltextSession session = fulltextStore.StartSession())
{
    session.Save(new Book
    {
        Id = 1,
        Author = "George R.R. Martin",
        Title = "A Game of Thrones: A Song of Ice and Fire: Book One",
        EbookAvailable = true,
        Categories = new long[] { 1, 2 },
        Price = 5.60m,
        ReleaseDate = new DateTime(1997, 8, 4)
     });
     
     session.FlushChanges();
}

To delete one or more documents from a real-time index you can pass their id's to the Delete method or an instance of a document you would like to delete:

using (IFulltextSession session = fulltextStore.StartSession())
{
    session.Delete<Book>(1, 2, 3);
    
    session.FlushChanges();
}

As you can see from the examples, you call FlushChanges to let SphinxConnector.NET know that the changes you made should be made persistent.

Automatic Optimization and Batching

When you call FlushChanges, all pending saves and deletes are gathered and executed in the most efficient manner. For the example above, that means that all deletes are executed in a single DELETE statement thus avoiding unnecessary network round-trips.

When SphinxConnector.NET detects that it needs to save more than one document, it inserts them in batches by generating a single REPLACE statement for each batch. This leads to a speed-up by several orders of magnitude compared to inserting documents one by one. The batch size is of course configurable, so you can fine tune it to your workload.

Back to top

Convention over Configuration

SphinxConnector.NET uses a convention based approach for many of its configuration settings. For example, when it comes to mapping objects to indexes or properties to attributes, you do not have to provide any manual mappings or clutter your code with custom attributes. The mapping is done via conventions that are defined in the FulltextStore class. You can of course change the default conventions if necessary.

Example for an index name convention for a main-delta setup:
IFulltextStore fulltextStore = new FulltextStore().Initialize();
fulltextStore.Conventions.GetIndexName = (type, ctx) => String.Format("{0}, {0}_delta", type.Name.ToLowerInvariant() + "s");

Notice that the conventions are defined via delegates. Often you have to implement interfaces or inherit from base classes to change some behavior in a component that you are using. We think that most of the time, this poses an unnecessary overhead because the customizations are just a few lines of code. In these cases providing a simple delegate saves time, keeps the code clean and easy to maintain. And, in more complex scenarios, you have the option to implement abstractions the way you see fit without being limited or constrained in your design by some 3rd party interfaces.

Back to top

Object Mapping for SphinxQL

If needed, you can also execute SphinxQL queries directly and have SphinxConnector.NET map the results to your document objects:

Example

using (IFulltextSession fulltextSession = fulltextStore.StartSession())
{
    ISphinxQLExecutor sphinxQLExecutor = fulltextSession.Advanced.CreateSphinxQLExecutor();

    var parameters = new { query = "a product", categories = new[] { 23, 42 } };

    var results = sphinxQLExecutor.Query<Product>("SELECT * FROM products WHERE MATCH(@query) AND categories IN @categories",                                                   parameters);        
}
Back to top

Designed for Testability

All functionality of the fluent API is exposed via interfaces. By applying techniques such as Inversion of Control and Dependency Injection, you can easily test your code by substituting these interfaces by mocks if necessary.

Using the fluent API with your favorite DI container is also straightforward:

Example

Configuring Castle Windsor for SphinxConnector.NET
public class SphinxConnectorInstaller : IWindsorInstaller
{
    public void Install(IWindsorContainer container, IConfigurationStore store)
    {
        container.Register(Component.For<IFulltextStore>().Instance(new FulltextStore().Initialize()).
                                     LifestyleSingleton(),
                           Component.For<IFulltextSession>().
                                     UsingFactoryMethod(kernel => kernel.Resolve<IFulltextStore>().StartSession()).
                                     LifestylePerWebRequest());
    }
}
Back to top

Classic API's

SphinxConnector.NET provides high performance implementations of the native API and standard ADO.NET 2.0 classes for executing SphinxQL statements. Please see below for a list of features exposed by each API:

Native API

  • Support for Sphinx 2.0.1 and up
  • Support for string attributes introduced with Sphinx 1.10.1
  • Support for features introduced in 0.9.9 like:
    • Persistent connections: open one connection for several operations to minimize network overhead.
    • Override attributes: temporarily change the value of an attribute without modifying the actual value for advanced search scenarios.
    • Select clause: write SQL-like statements to operate on attributes
  • Configurable Encoding
  • Search related features:
    • Set value and range filters
    • Schedule several queries for batch execution
    • Specify how Sphinx should match and rank documents
    • and more
  • Access to additional functions exposed by the Sphinx search engine:
    • Build excerpts
    • Build keywords
    • Update attributes
    • Query Sphinx for status variables
  • SphinxConnector.NET uses custom types as arguments for methods operating on Sphinx attributes. This gives you strongly typed access to these methods and also enables the compiler to check whether the operation is valid for a given type.

SphinxQL

  • Based on standard ADO.NET 2.0 classes:
    • SphinxQLConnection
    • SphinxQLConnectionStringBuilder
    • SphinxQLCommand
    • SphinxQLDataAdapter
    • SphinxQLDataReader
    • SphinxQLParameter
    • SphinxQLTransaction
  • Tailored to Sphinx specific data types and features, e.g.
    • Automatic conversion from .NET types to Sphinx types and vice versa
    • Methods for handling MVA values
  • Insert and Update Records in Real-Time Indexes
  • Select from any Index Type
  • Select via DataReader or DataAdapter
  • Connection Pooling for Maximum Performance
  • Support for Transactions
  • Support for TransactionScope (Local and Distributed Transactions)
  • Support for Command Parameters
  • Automatically detects and uses features based on Sphinx version, e.g. server side multi-query support with Sphinx 2.0.1 and above
Back to top

Logging

SphinxConnector.NET supports logging different types of messages to help developers identify problems or optimize their setup. By leveraging the well-known Common.Logging library, the logging infrastructure integrates with the most commonly used .NET logging frameworks including NLog, log4net and the Microsoft Enterprise Library.

Supported Platforms

SphinxConnector.NET needs at least .NET 3.5 to run and supports Sphinx 2.0.1 and up. With the latest release, experimental support for Mono has been introduced.

In case you need to access a Sphinx version < 2.0.1 or cannot use .NET 3.5, you can still download SphinxConnector.NET V2 which supports running under .NET 2.0 and supports Sphinx 0.98 - 2.0.4.

What does "experimental" with regards to Mono support mean?

We labeled the Mono support as being "experimental", so that users are aware that they might run into problems or issues when using SphinxConnector.NET with Mono and are encouraged to let us know about them. However, the Mono version passes the same unit and integration tests as the .NET version without problems. That makes us pretty confident that there won't be many major issues, but we want to let customers know that they should keep an extra eye out for possible issues, hence Mono support is currently "experimental". You are of course also encouraged to tell us if everythings works fine!