SphinxConnector.NET 4.0 has been released

by Dennis 18. April 2018 13:32

We’re pleased to announce that the next major release of SphinxConnector.NET is finally available for download and via NuGet!

Highlights

  • Support for .NET Core via .NET Standard 2.0
  • Fully asynchronous implementations of both SphinxQL and fluent API
  • Numerous performance improvements

In addition, Manticore Search is now fully supported without the need to change the mysql_version_string setting.

Please note that in order to use the .NET Standard 2.0 build, you’ll have to install SphinxConnector.NET via NuGet.

Breaking changes

A new method named Metadata() has been added to IFulltextQuery which returns query metadata via an out parameter. Overloads of ToList(), First() etc. that previously returned query metadata that way have been removed.

JSON.NET is now an explicit dependency, previously it had been merged into the SphinxConnector.NET assembly.

We've also decided to not provide a special build for Mono anymore, as demand for this was virtually nonexistent.

For a full list of changes please refer to the version history.

Tags:

Performance Improvements in SphinxConnector.NET 4.0

by Dennis 23. March 2018 14:02

As we're preparing the release of SphinxConnector.NET 4.0 I'd like to share some improvements we have made since V3. Lets start with performance: as many of you might be aware, there currently is a strong focus in the .NET ecosystem on improving performance, often done by reducing memory consumption, especially in low-level code. Case in point: .NET Core. The development of .NET Core is driven by a strong focus on providing great performance. This is accomplished, among other things, by using less memory to eliminate unnecessary garbage that would need to be collected by the GC. This is especially true for the aforementioned low-level stuff that might be called several hundred or even thousand times a second,e.g. in a very busy web application or API.

During the development of SphinxConnector.NET we took it upon ourselves to reduce the memory that is being used by the fluent API and SphinxQL. Following are two benchmarks (created with the great BenchmarkDotNet) based on our Stackoverflow with Sphinx sample web app. The benchmarked methods are based on the search method from the sample, which executes three queries: the search itself and two facet queries, so it resembles a scenario which is akin to what is done in a real-world application. The same queries are executed via plain SphinxQL and the fluent API. The benchmark is run against Manticore Search 2.6.2 on a Ubuntu Server 16 VM with 1 GB RAM and 1 CPU core, the search term used is ‘c#’.

BenchmarkDotNet=v0.10.12, OS=Windows 7 SP1 (6.1.7601.0)
Intel Core i5-3570 CPU 3.40GHz (Ivy Bridge), 1 CPU, 4 logical cores and 4 physical cores
Frequency=3330156 Hz, Resolution=300.2862 ns, Timer=TSC
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 32bit LegacyJIT-v4.7.2558.0
  Job-MZUNZG : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2558.0
Jit=RyuJit  Platform=X64  Force=False  
MethodMeanErrorStdDevOp/sGen 0Allocated
SearchSphinxQL 7.421 ms 0.1472 ms 0.1305 ms 134.76 23.4375 72.07 KB
SearchFluentAPI 10.555 ms 0.2626 ms 0.2579 ms 94.75 93.7500 314.95 KB

 

As you can see, there’s significantly less memory being allocated when queries are executed directly via SphinxQL compared to the fluent API. Some of this is of course expected, as the fluent API does the heavy lifting of creating the queries and the result objects for you, which comes with a cost. But there’s certainly room for improvement as you can see from the following results with SphinxConnector.NET 4.0:

MethodMeanErrorStdDevOp/sGen 0Allocated
SearchSphinxQL 7.552 ms 0.1415 ms 0.2441 ms 132.4 7.8125 45.99 KB
SearchFluentAPI 9.656 ms 0.1931 ms 0.4030 ms 103.6 31.2500 131.81 KB

 

Memory usage with SphinxQL has been reduced by about a third and Gen0 collections are down by over 60%! The reduction seen with the fluent API is even higher with more than 50% of allocations now gone and Gen0 collections reduced to a third of their previous value!

So what did we do to achieve these numbers? In this release we focused on addressing all the obvious issues that popped up under profiling and manual code inspections: avoid unnecessary copying of data, avoid or defer the creation of objects when possible, reuse buffers etc. This mostly applies to the SphinxQL infrastructure which needed a major overhaul anyway, to provide async implementations of query methods.

One thing that had a major impact on the fluent API, is the use of the awesome FastExpressionCompiler to compile expressions. It not only compiles expression trees faster than Expression.Compile() but also often produces faster code, which both results in an increase of Op/s.

To conclude, SphinxConnector.NET 4.0 comes with major improvements in this area, but there’s still room for more optimizations, which will be done over the next releases, so stay tuned!

Tags: ,

Using SphinxConnector.NET with Manticore Search

by Dennis 19. October 2017 11:04

As some of you might already know, former members of the Sphinx team have created a fork of Sphinx called Manticore Search. As Manticore Search uses a different version string format than Sphinx, SphinxConnector.NET’s version string parser currently fails to parse it. As a workaround, just set mysql_version_string to 2.4.1-id64 in your config.

This will be addressed in V4 of SphinxConnector.NET which is currently in development.

Tags:

SphinxConnector.NET 3.12 has been released

by Dennis 15. June 2016 10:26

We're pleased to announce that SphinxConnector.NET 3.12 is available for download and via NuGet!

The fluent API has gotten support for the new suggestion feature introduced in the current development branch of Sphinx, memory allocations during logging have been optimized and the SphinxHelper.EscapeString method now also escapes MAYBE.

Additionally, two bugs have been fixed. For details, please refer to the version history.

Tags:

SphinxConnector.NET 3.11 has been released

by Dennis 8. September 2015 11:48

We're pleased to announce that SphinxConnector.NET 3.11 is available for download and via NuGet!

Probably the most important change is an updated Sphinx version string parser, now that Sphinx has moved to Git. Before, the Sphinx version string would contain the SVN revision, which SphinxConnector.NET used (in addition to the version) to determine Sphinx capabilities (and certain quirks). Starting from 2.2.10, only the version will be used, previous versions build with the SVN revision will be treated as before, i.e. the SVN revision will be taken into account.

To see the rest of the changes, please refer to the version history.

Tags: