Ayende @ Rahien

Hi!
My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

, @ Q c

Posts: 5,953 | Comments: 44,409

filter by tags archive

NuGet Perf, The Final Part – Load Testing – Setup


So, after talking so long about the perf issues, here is the final part of this series. In which we actually take this for a spin using Load Testing.

I built a Web API application to serve as the test bed. It has a RavenController, which looks like this:

public class RavenController : ApiController
{
    private static IDocumentStore documentStore;

    public static IDocumentStore DocumentStore
    {
        get
        {
            if (documentStore == null)
            {
                lock (typeof (RavenController))
                {
                    if (documentStore != null)
                        return documentStore;
                    documentStore = new DocumentStore
                        {
                            Url = "http://localhost:8080",
                            DefaultDatabase = "Nuget"
                        }.Initialize();
                    IndexCreation.CreateIndexes(typeof (Packages_Search).Assembly, documentStore);
                }
            }
            return documentStore;
        }
    }

    public IDocumentSession DocumentSession { get; set; }

    public override async Task<HttpResponseMessage> ExecuteAsync(HttpControllerContext controllerContext, CancellationToken cancellationToken)
    {
        using (DocumentSession = DocumentStore.OpenSession())
        {
            HttpResponseMessage result = await base.ExecuteAsync(controllerContext, cancellationToken);
            DocumentSession.SaveChanges();
            return result;
        }
    }
}

And now we have the following controllers:

public class PackagesController : RavenController
{
    public IEnumerable<Packages_Search.ReduceResult> Get(int page = 0)
    {
        return DocumentSession.Query<Packages_Search.ReduceResult, Packages_Search>()
            .Where(x=>x.IsPrerelease == false)
            .OrderByDescending(x=>x.DownloadCount)
                .ThenBy(x=>x.Created)
            .Skip(page*30)
            .Take(30)
            .ToList();
    }
}

public class SearchController : RavenController
{
    public IEnumerable<Packages_Search.ReduceResult> Get(string q, int page = 0)
    {
        return DocumentSession.Query<Packages_Search.ReduceResult, Packages_Search>()
            .Search(x => x.Query, q)
            .Where(x => x.IsPrerelease == false)
            .OrderByDescending(x => x.DownloadCount)
                .ThenBy(x => x.Created)
            .Skip(page * 30)
            .Take(30)
            .ToList();
    }
}

And, just for completeness sake, the Packages_Search index looks like this:

public class Packages_Search : AbstractIndexCreationTask<Package, Packages_Search.ReduceResult>
{
    public class ReduceResult
    {
        public DateTime Created { get; set; }
        public int DownloadCount { get; set; }
        public string PackageId { get; set; }
        public bool IsPrerelease { get; set; }
        public object[] Query { get; set; }
    }

    public Packages_Search()
    {
        Map = packages => from p in packages
                          select new
                              {
                                  p.Created, 
                                  DownloadCount = p.VersionDownloadCount, 
                                  p.PackageId, 
                                  p.IsPrerelease,
                                  Query = new object[] { p.Tags, p.Title, p.PackageId}
                              };
        Reduce = results =>
                 from result in results
                 group result by new {result.PackageId, result.IsPrerelease}
                 into g
                 select new
                         {
                             g.Key.PackageId,
                             g.Key.IsPrerelease,
                             DownloadCount = g.Sum(x => x.DownloadCount),
                             Created = g.Select(x => x.Created).OrderBy(x => x).First(),
                             Query = g.SelectMany(x=>x.Query).Distinct()
                         };

        Store(x=>x.Query, FieldStorage.No);
    }
}

That is enough setup, in the next post, I’ll discuss the actual structure of the load tests.


Comments

Simon Hughes

Best not to use lock (typeof (RavenController)), as some other code could also lock on that type. Better to use a private lock field instead.

What about using Lazy instead?

Simon Hughes

hmm, my less than and greater than were removed. Lazy < DocumentStore >

Adam Ralph

+1 Simon Hughes... never lock on something which other peoples code can see.

Chris

I'll echo the other guys about the lock(typeof(...)). Locking on a Type instance is quite dangerous as the same instance can be used across AppDomains. The documentation (http://msdn.microsoft.com/en-us/library/c5kehkcz(v=vs.80).aspx) says that it is only a problem if the Type is publicly accessible, but it seems to me that it could also be a problem if someone got the Type instance through reflection or returned polymorphically through an interface or something.

Here is an analysis of the problem: http://www.pcreview.co.uk/forums/dont-lock-type-objects-t1373491.html (this is from 2004, so it may not be completely accurate anymore).

Matt Johnson

Ayende, I thought the general guidance was to initialize the documentstore during application startup in global.asax? The way you have it here, there will be a delay when the first user hits the controller.

Also, this creates a separate documentstore for each controller, which I also thought was bad form. Shouldn't there just be one per app?

Chanan Braunstein

Ayende, What is the purpose of Store(x=>x.Query, FieldStorage.No);? What does it do if it is not storing the query field in the index? When should it be used, etc...

João P. Bragança

What other code is actually going to lock on that type? this is not production code, it's for a PERF TEST. Good lord.

tatabánya gyerekangol

@chanan braunstein: may it's not just about fields...but I'm courious too:)

Simon

@Matt The document store is static, so there will only be one per app not one for each controller. The delay is effectively the same as in Application_Start, as that won't fire until the first request anyway (assuming not using the new "warm-up" app thingy added in ASP.NET 4, and that the first request will end up hitting a controller).

Matt Warren

@Chanan Braunstein, @tatabánya gyerekangol

For a MapReduce index the default is FieldStorage.Yes, so this is just an optimisation.

You can still query against a field when it has FieldStorage.No, but it means that you just can't read that value from the index. So in this case reading the contents of "Query" field is not needed, so not storing it saves space.

Matt Johnson

@Simon - Thanks for pointing that out. I was thinking about the problem of static fields in GENERIC classes. You are right, there will only be one static documentstore instance here, so I guess it's ok.

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

No future posts left, oh my!

RECENT SERIES

  1. The RavenDB Comic Strip (3):
    28 May 2015 - Part III – High availability & sleeping soundly
  2. Special Offer (2):
    27 May 2015 - 29% discount for all our products
  3. RavenDB Sharding (3):
    22 May 2015 - Adding a new shard to an existing cluster, splitting the shard
  4. Challenge (45):
    28 Apr 2015 - What is the meaning of this change?
  5. Interview question (2):
    30 Mar 2015 - fix the index
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats