Speeding up NuGet.Server with Lucene

Topics: General
May 10, 2012 at 9:44 PM

Recently I've been working on extending NuGet.Server to integrate the package feed with Lucene.Net, and I think my changes are stable enough to request feedback from the community and propose the possibility of having them merged into origin.

You can read a blurb about the project here: https://github.com/themotleyfool/NuGet

Or download the binaries to test it out here: https://github.com/themotleyfool/NuGet/downloads

I'm soliciting feedback on my changes and asking what, if anything, I can improve or modify to get these changes pulled.  I wanted to start a discussion before actually submitting the pull request since on the whole the pull request will be substantial.



May 15, 2012 at 7:31 AM
Edited May 15, 2012 at 7:39 AM

Hi Chris, i've downloaded your binaries, fired up the website, and I'm getting errors like the one below,

my guess is the lucene indexes need to be initialized. But how?

type="System.IO.FileNotFoundException" message="no segments* file found in Lucene.Net.Store.SimpleFSDirectory@D:\nuget\lucene lockFactory=Lucene.Net.Store.NativeFSLockFactory: files: write.lock" source="mscorlib" detail="System.IO.FileNotFoundException: no segments* file found in Lucene.Net.Store.SimpleFSDirectory@D:\nuget\lucene lockFactory=Lucene.Net.Store.NativeFSLockFactory: files: write.lock Server stack trace: at Lucene.Net.Index.SegmentInfos.FindSegmentsFile.Run(IndexCommit commit) at Lucene.Net.Index.SegmentInfos.Read(Directory directory) at Lucene.Net.Index.IndexWriter.Init(Directory d, Analyzer a, Boolean create, Boolean

Also, the search button assumes the nuget website is directly at the server root.

It redirects me to https://localhost/api/v2/Search()?includePrerelease=false&targetFramework=''&searchTerm=''

but the nuget service is rooted at https://localhost/Updates

May 15, 2012 at 8:28 AM

The index problem has been solved, I tried to be smart and created the lucene index directory in advance, this caused the initialization code to think the index was already there.

Maybe it would be better to check for both the existence of the index directory and any files in index directory, to make it more fool-proof?

May 15, 2012 at 12:26 PM
Edited May 15, 2012 at 12:27 PM

I've got this git thingy working now, so there is a pull request waiting to fix the issues i mentioned above

May 15, 2012 at 2:08 PM

Thanks for the tweaks. I noticed that issue with an empty index folder but hadn't gotten around to fixing it.

If I have time I'm planning on implementing a better UI for search and formatting the results.

By the way, if anyone else feels like working on this, please base your changes off the lucene branch, not master.  The lucene branch is based off nuget 1.7 so it should be considered more stable than master. I've been merging the changes into master as well to keep everything in sync.

Feb 1, 2013 at 10:56 PM
Hi, I'm new on the nuget scene, and am curious what happened with this.
It looks like current web gallery code is lucene based, but it seems to be divergent from this fork, which was of server, which I guess is yet another fork of gallery code?
Feb 2, 2013 at 11:56 PM
The NuGet Gallery that powers nuget.org is a separate project with a separate codebase on github.com. Having looked at the code, it does appear to be using Lucene.Net, but in a different capacity than the project I started. My fork is still alive at https://github.com/themotleyfool/NuGet, and I'm working on splitting it out from the NuGet repository into a separate codebase (I don't see it getting merged into NuGet.Server, so no reason to keep it integrated with that code). I'm planning on having a package on http://chocolatey.org/ to help people stand up private package feeds easily.