A Quick Way to Setup Logging during Development

By Dennis

I’ve been asked a few times if there’s a quick way to get logging output from SphinxConnector.NET without setting up a “real” logging framework like NLog. Here’s one: Common.Logging comes with two adapters named TraceLoggerFactoryAdapter and ConsoleOutLoggerFactoryAdapter. The latter (obviously) logs messages to the console, while the former logs messages via .NET’s Trace class. One nice thing about the trace log is that it can be accessed via Visual Studio’s ‘Output’ window (CTRL+ALT+O) if your application is running with a debugger attached (F5).

Here is the relevant code:

[Conditional("DEBUG")]
private static void SetupLogging()
{
    LogManager.Adapter = new TraceLoggerFactoryAdapter
    {
        Level = LogLevel.All,
        ShowLevel = true,
        ShowDateTime = true,
    };
}

I also added a Conditional attribute to the setup method to ensure that it is only being called in a debug configuration.

Using SphinxConnector.NET with ASP.NET MVC

By Dennis

As using Sphinx from a web application is probably the most common use case,  I thought I’d post some guidelines and examples on how to use the fluent API in an ASP.NET MVC application with regards to setup and proper handling of IFulltextStore and IFulltextSession. The documentation already mentions that there should (usually) be one instance of the FulltextStore per application and one IFulltextSession per thread/(web-) request. Let’s take a look at a few different approaches to this:

Using Lazy

This approach makes use of the Lazy<T> class that was introduced with .NET 4.0. We create a base controller that holds the IFulltextStore instance which will be initialized upon the first access. Lazy<T> will make sure that the FulltextStore is created only once in a thread-safe way.

public abstract class SearchController : Controller
{
    private static readonly Lazy<IFulltextStore> Store = new Lazy<IFulltextStore>(() =>
    {
        IFulltextStore fulltextStore = new FulltextStore().Initialize();
        fulltextStore.ConnectionString.IsThis("pooling=true");

        return fulltextStore;
    });

    protected static IFulltextStore FulltextStore
    {
        get { return Store.Value; }
    }

    protected IFulltextSession FulltextSession { get; private set; }

    protected override void OnActionExecuting(ActionExecutingContext filterContext)
    {
        FulltextSession = FulltextStore.StartSession();
    }

    protected override void OnActionExecuted(ActionExecutedContext filterContext)
    {
        if (filterContext.IsChildAction || FulltextSession == null)
            return;

        using (FulltextSession)
        {
            if (filterContext.Exception != null)
                return;

            FulltextSession.FlushChanges();
        }
    }
}

The IFulltextSession for every request is created in an override of OnActionExecuting by assigning the result of StartSession to the FulltextSession property. This way, every controller that inherits from SearchController automatically gets an open session that is ready for use. In the override of OnActionExecuted we tell the FulltextSession to flush all pending changes. The using statement ensures that it is properly disposed of.

Using an IoC-Container

Following is an example installer for Castle Windsor:

public class SphinxConnectorInstaller : IWindsorInstaller
{
    public void Install(IWindsorContainer container, IConfigurationStore store)
    {
        container.Register(Component.For<IFulltextStore>().
                                     Instance(new FulltextStore().Initialize()).
                                     LifestyleSingleton(),
                           Component.For<IFulltextSession>().
                                     UsingFactoryMethod(kernel =>
                                         kernel.Resolve<IFulltextStore>().StartSession()).
                                     LifestylePerWebRequest());
    }
}

In this example, we setup Castle Windsor so that it can create both IFulltextStore and IFulltextSession. If you wanted to create IFulltextSession yourself (by injecting IFulltextStore into your classes and calling StartSession), you could remove the corresponding code from the installer.

We instruct Windsor to use the Singleton lifestyle for IFulltextStore, which means that Windsor will create one instance per container*.* In fact, Windsor uses Singleton is the default lifestyle, but in cases like this I’d like to make that explicit, so that developers that are not familiar with Windsor immediately see what’s going on. For IFulltextSession we set LifestylePerWebRequest so that Windsor will create an instance for each request; it will also automatically call Dispose at the end of each request, so we don’t have to worry about that. If you wanted Windsor to also call FlushChanges, you could do so with the help of Windsor’s OnDestroy method.

Initialization at Application Startup

Like with the first approach, we create a base controller, this time with a static property hat holds the IFulltextStore instance. The instance is initialized in the Global.asax.cs file in Application_Start:

public abstract class SearchController : Controller 
{
    public static IFulltextStore FulltextStore { get; set; }

    protected IFulltextSession FulltextSession { get; private set; } 
    
    //Overrides of OnActionExecuting and OnActionExecuted omitted
}
protected void Application_Start()
{
    AreaRegistration.RegisterAllAreas();

    RegisterGlobalFilters(GlobalFilters.Filters);
    RegisterRoutes(RouteTable.Routes);

    InitFulltextStore();
}

private static void InitFulltextStore()
{
    IFulltextStore fulltextStore = new FulltextStore().Initialize();
    fulltextStore.ConnectionString.IsThis("pooling=true");

    SearchController.FulltextStore = fulltextStore;
}

Importing Data into Sphinx RT-Indexes with SphinxConnector.NET’s Fluent API

By Dennis

If you are facing the task of importing data into a Sphinx RT-index, SphinxConnector.NET’s fluent API makes this really easy with just a couple lines of code (the document model class is omitted for brevity):

void Import()
{
    IFulltextStore fulltextStore = new FulltextStore().Initialize();
    fulltextStore.ConnectionString.IsThis("pooling=true");

    int count = 0;

    using (IFulltextSession session = fulltextStore.StartSession())
    {
        foreach (var document in GetDocuments())
        {
            session.Save(document);

            if (++count % fulltextStore.Settings.SaveBatchSize == 0)
                session.FlushChanges();
        }

        session.FlushChanges();   
    }
}   

The important part is the call to FlushChanges each time a batch of documents has been passed to Save. This avoids high memory usage when importing many documents, because SphinxConnector.NET has to keep each document in memory until FlushChanges is called (though for smaller datasets it might be acceptable to flush all changes at the end of the import process).

Not only is this much simpler than writing SphinxQL by hand, it’s also faster because of SphinxConnector.NET’s automatic batching. The default value for SaveBatchSize is 16, which provides good performance, but can of course be adjusted for environments where a higher batch size leads to even more performance.

SphinxConnector.NET 3.1.3 released

By Dennis

SphinxConnector.NET 3.1.3 has just been made available for download and via NuGet. A list of resolved issues is available in the version history.

SphinxConnector.NET 3.1 has been released

By Dennis

We're pleased to announce the immediate availability of SphinxConnector.NET 3.1!

This version introduces support for future queries with the fluent query API. This feature allows to defer the execution of queries until their results are needed, enabling SphinxConnector.NET to send them in batches. This functionality is available with .NET 4.0 or higher and Mono.

Additional improvements to the fluent query API:

  • IFulltextQuery.Where can now be called multiple times for a query.
  • The generic type of collections for multi-value attributes e.g. IList<T> etc. can now be of any type that is large enough to hold the result values. Previously only 'long' was supported.
  • A workaround to the query generator has been added, to account for the fact that Sphinx currently doesn't support using 'weight()' in an expression, e.g. 'SELECT weight()*2 AS c1 FROM example'. The query generator will now emit a separate, aliased weight() column and use the alias in the expression, e.g. 'SELECT weight() AS c2, c2*2 AS c1 FROM example'.

Last but not least, a bug that caused the RecordsAffected property of the SphinxQLDataReader class not being set correctly when the reader was used to read results from multiple SELECT or SHOW statements, has been fixed.