3. Querying


db4o supplies three querying systems, Query-By-Example (QBE) Native Queries (NQ), and the SODA API. In the previous chapter, you were briefly introduced to Query By Example(QBE).

Query-By-Example (QBE) is appropriate as a quick start for users who are still acclimating to storing and retrieving objects with db4o.

Native Queries (NQ) are the main db4o query interface, recommended for general use.

SODA is the underlying internal API. It is provided for backward compatibility and it can be useful for dynamic generation of queries, where NQ are too strongly typed.



    3.1. Query by Example (QBE)


    When using Query By Example (QBE) you provide db4o with a template object. db4o will return all of the objects which match all non-default field values. This is done via reflecting all of the fields and building a query expression where all non-default-value fields are combined with AND expressions. Here's an example from the previous chapter:

    // retrievePilotByName
    Pilot proto = new Pilot("Michael Schumacher", 0);
    IObjectSet result = db.QueryByExample(proto);
    ListResult(result);


    Querying this way has some obvious limitations:
    - db4o must reflect all members of your example object.
    - You cannot perform advanced query expressions. (AND, OR, NOT, etc.)
    - You cannot constrain on values like 0 (integers), "" (empty strings), or nulls (reference types) because they would be interpreted as unconstrained.
    - You need to be able to create objects without initialized fields. That means you can not initialize fields where they are declared. You can not enforce contracts that objects of a class are only allowed in a well-defined initialized state.
    - You need a constructor to create objects without initialized fields.

    To get around all of these constraints, db4o provides the Native Query (NQ) system.




    3.2. Native Queries


    Wouldn't it be nice to pose queries in the programming language that you are using? Wouldn't it be nice if all your query code was 100% typesafe, 100% compile-time checked and 100% refactorable? Wouldn't it be nice if the full power of object-orientation could be used by calling methods from within queries? Enter Native Queries.

    Native queries are the main db4o query interface and they are the recommended way to query databases from your application. Because native queries simply use the semantics of your programming language, they are perfectly standardized and a safe choice for the future.

    Native Queries are available for all platforms supported by db4o.


      3.2.1. Concept

      The concept of native queries is taken from the following two papers:

      - Cook/Rosenberger, Native Queries for Persistent Objects, A Design White Paper
      - Cook/Rai, Safe Query Objects: Statically Typed Objects as Remotely Executable Queries


      3.2.2. Principle

      Native Queries provide the ability to run one or more lines of code against all instances of a class. Native query expressions should return true to mark specific instances as part of the result set. db4o will attempt to optimize native query expressions and run them against indexes and without instantiating actual objects, where this is possible.


      3.2.3. Simple Example

      Let's look at how a simple native query will look like in some of the programming languages and dialects that db4o supports:

      C# .NET 2.0
      IList <Pilot> pilots = db.Query <Pilot> (delegate(Pilot pilot) {
          return pilot.Points == 100;
      });
       

      Java JDK 5
      List <Pilot> pilots = db.query(new Predicate<Pilot>() {
          public boolean match(Pilot pilot) {
              return pilot.getPoints() == 100;
          }
      });


      Java JDK 1.2 to 1.4
      List pilots = db.query(new Predicate() {
          public boolean match(Pilot pilot) {
              return pilot.getPoints() == 100;
          }
      });


      Java JDK 1.1
      ObjectSet pilots = db.query(new PilotHundredPoints());

      public static class PilotHundredPoints extends Predicate {
          public boolean match(Pilot pilot) {
              return pilot.getPoints() == 100;
          }
      }


      C# .NET 1.1
      IList pilots = db.Query(new PilotHundredPoints());

      public class PilotHundredPoints : Predicate {
          public boolean Match(Pilot pilot) {
              return pilot.Points == 100;
          }
      }


      VB .NET 1.1
      Dim pilots As IList = db.Query(new PilotHundredPoints())

      Public Class PilotHundredPoints
          Inherits Predicate
          Public Function Match (pilot As Pilot) as Boolean
              If pilot.Points = 100 Then
                  Return True
              Else
                  Return False
          End Function
      End Class


      A side note on the above syntax:
      For all dialects without support for generics, Native Queries work by convention. A class that extends the com.db4o.Predicate class is expected to have a boolean #Match() method with one parameter to describe the class extent:

       
      bool Match(Pilot candidate);
        

      When using native queries, don't forget that modern integrated development environments (IDEs) can do all the typing work around the native query expression for you, if you use templates and autocompletion.

      The following example shows how to create an autocompletion code snippet in Visual Studio 2005.
      Create a "nq.snippet" file in any text editor.
      Paste the following code:

       <?xml version="1.0" encoding="utf-8" ?>
      <CodeSnippets  xmlns="http://schemas.microsoft.com/VisualStudio/2005/CodeSnippet">
          <CodeSnippet Format="1.0.0">
              <Header>
                  <Title>NQ</Title>
                  <Shortcut>NQ</Shortcut>
                  <Description>Code snippet for Native Query</Description>
                  <Author>db4objects Inc.</Author>
                  <SnippetTypes>
                      <SnippetType>Expansion</SnippetType>
                  </SnippetTypes>
              </Header>
              <Snippet>
                  <Declarations>
                      <Literal>
                          <ID>type</ID>
                          <ToolTip>Type</ToolTip>
                          <Default>MyType</Default>
                      </Literal>
                      <Literal>
                          <ID>var</ID>
                          <ToolTip>variable</ToolTip>
                          <Default>myType</Default>
                      </Literal>
                      <Literal>
                          <ID>expression</ID>
                          <ToolTip>Boolean expression</ToolTip>
                          <Default>return true;</Default>
                      </Literal>
                  </Declarations>
                  <Code Language="csharp"><![CDATA[ IList<$type$> list = db.Query<$type$>(delegate($type$ $var$)
                      {
                          $expression$
                      });]]>
                  </Code>
              </Snippet>
          </CodeSnippet>
      </CodeSnippets>
       

      Save the file.
      In the Visual Studio 2005 open Tools/Code SnippetsManager
      Select language "Visual c#" if not yet selected. Press "Import..." button and select the newly created file. Select "Visual c#" as the location and press "Finish" button.
      Now you can use the snippet by selecting it from "Edit/InstelliSense/Insert Snippet..." menu.



      3.2.4. Advanced Example

      For complex queries, the native syntax is very precise and quick to write. Let's compare to a SODA query that finds all pilots with a given name or a score within a given range:

      // storePilots
      db.Store(new Pilot("Michael Schumacher", 100));
      db.Store(new Pilot("Rubens Barrichello", 99));


      // retrieveComplexSODA
      IQuery query=db.Query();
      query.Constrain(typeof(Pilot));
      IQuery pointQuery=query.Descend("_points");
      query.Descend("_name").Constrain("Rubens Barrichello")
          .Or(pointQuery.Constrain(99).Greater()
          .And(pointQuery.Constrain(199).Smaller()));
      IObjectSet result=query.Execute();
      ListResult(result);


      Here is how the same query will look like with native query syntax, fully accessible to autocompletion, refactoring and other IDE features, fully checked at compile time:

      C# .NET 2.0
      IList <Pilot> result = db.Query<Pilot> (delegate(Pilot pilot) {
          return pilot.Points > 99
              && pilot.Points < 199
              || pilot.Name == "Rubens Barrichello";
      });
       

      Java JDK 5
      List <Pilot> result = db.query(new Predicate<Pilot>() {
          public boolean match(Pilot pilot) {
              return pilot.getPoints() > 99
                  && pilot.getPoints() < 199
                  || pilot.getName().equals("Rubens Barrichello");
         }
      });



      3.2.5. Arbitrary Code

      Basically that's all there is to know about native queries to be able to use them efficiently. In principle you can run arbitrary code as native queries, you just have to be very careful with side effects - especially those that might affect persistent objects.

      Let's run an example that involves some more of the language features available.
      using Db4objects.Db4o.Query;
      namespace Db4objects.Db4o.Tutorial.F1.Chapter1
      {
          public class ArbitraryQuery : Predicate
          {
              private int[] _points;
              
              public ArbitraryQuery(int[] points)
              {
                  _points=points;
              }
          
              public bool Match(Pilot pilot)
              {
                  foreach (int points in _points)
                  {
                      if (pilot.Points == points)
                      {
                          return true;
                      }
                  }
                  return pilot.Name.StartsWith("Rubens");
              }
          }
      }




      3.2.6. Native Query Performance

      One drawback of native queries has to be pointed out: Under the hood db4o tries to analyze native queries to convert them to SODA. This is not possible for all queries. For some queries it is very difficult to analyze the flowgraph. In this case db4o will have to instantiate some of the persistent objects to actually run the native query code. db4o will try to analyze parts of native query expressions to keep object instantiation to the minimum.

      The development of the native query optimization processor will be an ongoing process in a close dialog with the db4o community. Feel free to contribute your results and your needs by providing feedback to our db4o forums.


      With the current implementation, all above examples will run optimized, except for the "Arbitrary Code" example - we are working on it.


      3.2.7. Full source


      using Db4objects.Db4o;
      using Db4objects.Db4o.Query;
      using Db4objects.Db4o.Tutorial;
      namespace Db4objects.Db4o.Tutorial.F1.Chapter1
      {
          public class NQExample : Util
          {
              public static void Main(string[] args)
              {
                  IObjectContainer db = Db4oFactory.OpenFile(Util.YapFileName);
                  try
                  {
                      StorePilots(db);
                      RetrieveComplexSODA(db);
                      RetrieveComplexNQ(db);
                      RetrieveArbitraryCodeNQ(db);
                      ClearDatabase(db);
                  }
                  finally
                  {
                      db.Close();
                  }
              }
          
              public static void StorePilots(IObjectContainer db)
              {
                  db.Store(new Pilot("Michael Schumacher", 100));
                  db.Store(new Pilot("Rubens Barrichello", 99));
              }
          
              public static void RetrieveComplexSODA(IObjectContainer db)
              {
                  IQuery query=db.Query();
                  query.Constrain(typeof(Pilot));
                  IQuery pointQuery=query.Descend("_points");
                  query.Descend("_name").Constrain("Rubens Barrichello")
                      .Or(pointQuery.Constrain(99).Greater()
                      .And(pointQuery.Constrain(199).Smaller()));
                  IObjectSet result=query.Execute();
                  ListResult(result);
              }
              public static void RetrieveComplexNQ(IObjectContainer db)
              {
                  IObjectSet result = db.Query(new ComplexQuery());
                  ListResult(result);
              }
              public static void RetrieveArbitraryCodeNQ(IObjectContainer db)
              {
                  IObjectSet result = db.Query(new ArbitraryQuery(new int[]{1,100}));
                  ListResult(result);
              }
          
              public static void ClearDatabase(IObjectContainer db)
              {
                  IObjectSet result = db.QueryByExample(typeof(Pilot));
                  while (result.HasNext())
                  {
                      db.Delete(result.Next());
                  }
              }
          }
      }


      using Db4objects.Db4o.Query;
      namespace Db4objects.Db4o.Tutorial.F1.Chapter1
      {
          public class ComplexQuery : Predicate
          {
              public bool Match(Pilot pilot)
              {
                  return pilot.Points > 99
                      && pilot.Points < 199
                      || pilot.Name=="Rubens Barrichello";
              }
          }
      }

      using Db4objects.Db4o.Query;
      namespace Db4objects.Db4o.Tutorial.F1.Chapter1
      {
          public class ArbitraryQuery : Predicate
          {
              private int[] _points;
              
              public ArbitraryQuery(int[] points)
              {
                  _points=points;
              }
          
              public bool Match(Pilot pilot)
              {
                  foreach (int points in _points)
                  {
                      if (pilot.Points == points)
                      {
                          return true;
                      }
                  }
                  return pilot.Name.StartsWith("Rubens");
              }
          }
      }




    3.3. LINQ


    db4o querying syntax has got even easier with the introduction of .NET LINQ queries. LINQ allows you to write compile checked db4o queries, which can be refactored automatically when a field name changes and which are supported by code auto-completion tools.
    In order to use LINQ you will need to add reference to Db4objects.Db4o.Linq.dll and usage to your program class:
    using System.Linq;
    using Db4objects.Db4o.Linq;
     
    If you are already familiar with LINQ syntax, you can just start writing LINQ to query db4o.  Otherwise you may want to familiarise yourself with LINQ resources on MSDN


      3.3.1. Linq Examples

      Let's prepare some objects in our database to query against:
      // storeObjects
      db.Store(new Car("Ferrari", (new Pilot("Michael Schumacher", 100))));
      db.Store(new Car("BMW", (new Pilot("Rubens Barrichello", 99))));


      The simplest LINQ query will look like this:
      // retrievePilot
      IEnumerable<Pilot> result = from Pilot p in db
                                  where p.Name.StartsWith("Michael")
                                  select p;
      ListResult(result);

      You can see that we are using db4o object container as a datasource, the rest of the syntax is generic to all LINQ queries.

      Now let's try a bit more complex selection:
      // retrievePilotByCar
      IEnumerable<Pilot> result = from Car c in db
                                  where c.Model.StartsWith("F")
                                  && (c.Pilot.Points > 99 && c.Pilot.Points <150)
                                  select c.Pilot;
      ListResult(result);

      So we can constrain on one object and retrieve a list of others. You can even create completely new objects based on the retrieved information using select new MyObject(field1, field2...). Try to experiment with different LINQ queries against db4o database.


      3.3.2. Performance

      db4o query processor is based on SODA queries, therefore LINQ query is analysed and converted to SODA syntax in the runtime. However, in some cases this conversion is not possible. This can happen when query is constrained against aggregates or projections of a field value and in other cases when SODA equivalent does not exists. For example:
      // retrievePilotUnoptimized
      IEnumerable<Pilot> result = from Pilot p in db
                                  where (p.Points - 81) == p.Name.Length
                                  select p;
      ListResult(result);

      The query still works, but it requires instantiation of all candidate objects, which is much less performant than SODA query.






    3.4. SODA Query API


    The SODA query API is db4o's low level querying API, allowing direct access to nodes of query graphs. Since SODA uses strings to identify fields, it is neither perfectly typesafe nor compile-time checked and it also is quite verbose to write.

    For most applications Native Queries will be the better querying interface.

    However there can be applications where dynamic generation of queries is required, that's why SODA is explained here.


      3.4.1. Simple queries


      Let's see how our familiar QBE queries are expressed with SODA. A new Query object is created through the #Query() method of the ObjectContainer and we can add Constraint instances to it. To find all Pilot instances, we constrain the query with the Pilot class object.

      // retrieveAllPilots
      IQuery query = db.Query();
      query.Constrain(typeof(Pilot));
      IObjectSet result = query.Execute();
      ListResult(result);


      Basically, we are exchanging our 'real' prototype for a meta description of the objects we'd like to hunt down: a query graph made up of query nodes and constraints. A query node is a placeholder for a candidate object, a constraint decides whether to add or exclude candidates from the result.

      Our first simple graph looks like this.



      We're just asking any candidate object (here: any object in the database) to be of type Pilot to aggregate our result.

      To retrieve a pilot by name, we have to further constrain the candidate pilots by descending to their name field and constraining this with the respective candidate String.

      // retrievePilotByName
      IQuery query = db.Query();
      query.Constrain(typeof(Pilot));
      query.Descend("_name").Constrain("Michael Schumacher");
      IObjectSet result = query.Execute();
      ListResult(result);


      What does 'descend' mean here? Well, just as we did in our 'real' prototypes, we can attach constraints to child members of our candidates.



      So a candidate needs to be of type Pilot and have a member named 'name' that is equal to the given String to be accepted for the result.

      Note that the class constraint is not required: If we left it out, we would query for all objects that contain a 'name' member with the given value. In most cases this will not be the desired behavior, though.

      Finding a pilot by exact points is analogous.

      // retrievePilotByExactPoints
      IQuery query = db.Query();
      query.Constrain(typeof(Pilot));
      query.Descend("_points").Constrain(100);
      IObjectSet result = query.Execute();
      ListResult(result);



      3.4.2. Advanced queries


      Now there are occasions when we don't want to query for exact field values, but rather for value ranges, objects not containing given member values, etc. This functionality is provided by the Constraint API.

      First, let's negate a query to find all pilots who are not Michael Schumacher:

      // retrieveByNegation
      IQuery query = db.Query();
      query.Constrain(typeof(Pilot));
      query.Descend("_name").Constrain("Michael Schumacher").Not();
      IObjectSet result = query.Execute();
      ListResult(result);


      Where there is negation, the other boolean operators can't be too far.

      // retrieveByConjunction
      IQuery query = db.Query();
      query.Constrain(typeof(Pilot));
      IConstraint constr = query.Descend("_name")
              .Constrain("Michael Schumacher");
      query.Descend("_points")
              .Constrain(99).And(constr);
      IObjectSet result = query.Execute();
      ListResult(result);


      // retrieveByDisjunction
      IQuery query = db.Query();
      query.Constrain(typeof(Pilot));
      IConstraint constr = query.Descend("_name")
              .Constrain("Michael Schumacher");
      query.Descend("_points")
              .Constrain(99).Or(constr);
      IObjectSet result = query.Execute();
      ListResult(result);


      We can also constrain to a comparison with a given value.

      // retrieveByComparison
      IQuery query = db.Query();
      query.Constrain(typeof(Pilot));
      query.Descend("_points")
              .Constrain(99).Greater();
      IObjectSet result = query.Execute();
      ListResult(result);


      The query API also allows to query for field default values. 
      // retrieveByDefaultFieldValue
      Pilot somebody = new Pilot("Somebody else", 0);
      db.Store(somebody);
      IQuery query = db.Query();
      query.Constrain(typeof(Pilot));
      query.Descend("_points").Constrain(0);
      IObjectSet result = query.Execute();
      ListResult(result);
      db.Delete(somebody);


      It is also possible to have db4o sort the results.

      // retrieveSorted
      IQuery query = db.Query();
      query.Constrain(typeof(Pilot));
      query.Descend("_name").OrderAscending();
      IObjectSet result = query.Execute();
      ListResult(result);
      query.Descend("_name").OrderDescending();
      result = query.Execute();
      ListResult(result);


      All these techniques can be combined arbitrarily, of course. Please try it out. There still may be cases left where the predefined query API constraints may not be sufficient - don't worry, you can always let db4o run any arbitrary code that you provide in an Evaluation. Evaluations will be discussed in a later chapter.

      To prepare for the next chapter, let's clear the database.

      // clearDatabase
      IObjectSet result = db.QueryByExample(typeof(Pilot));
      foreach (object item in result)
      {
          db.Delete(item);
      }



      3.4.3. Conclusion

      Now you have been provided with three alternative approaches to query db4o databases: Query-By-Example, Native Queries, SODA.

      Which one is the best to use? Some hints:
      - Native queries are targetted to be the primary interface for db4o, so they should be preferred.
      - With the current state of the native query optimizer there may be queries that will execute faster in SODA style, so it can be used to tune applications. SODA can also be more convenient for constructing dynamic queries at runtime.
      - Query-By-Example is nice for simple one-liners, but restricted in functionality. If you like this approach, use it as long as it suits your application's needs.

      Of course you can mix these strategies as needed.

      We have finished our walkthrough and seen the various ways db4o provides to pose queries. But our domain model is not complex at all, consisting of one class only. Let's have a look at the way db4o handles object associations in the next chapter .


      3.4.4. Full source


      using System;
      using Db4objects.Db4o;
      using Db4objects.Db4o.Query;
      using Db4objects.Db4o.Tutorial;
      namespace Db4objects.Db4o.Tutorial.F1.Chapter1
      {
          public class QueryExample : Util
          {
              public static void Main(string[] args)
              {
                  IObjectContainer db = Db4oFactory.OpenFile(Util.YapFileName);
                  try
                  {
                      StoreFirstPilot(db);
                      StoreSecondPilot(db);
                      RetrieveAllPilots(db);
                      RetrievePilotByName(db);
                      RetrievePilotByExactPoints(db);
                      RetrieveByNegation(db);
                      RetrieveByConjunction(db);
                      RetrieveByDisjunction(db);
                      RetrieveByComparison(db);
                      RetrieveByDefaultFieldValue(db);
                      RetrieveSorted(db);
                      ClearDatabase(db);
                  }
                  finally
                  {
                      db.Close();
                  }
              }
          
              public static void StoreFirstPilot(IObjectContainer db)
              {
                  Pilot pilot1 = new Pilot("Michael Schumacher", 100);
                  db.Store(pilot1);
                  Console.WriteLine("Stored {0}", pilot1);
              }
          
              public static void StoreSecondPilot(IObjectContainer db)
              {
                  Pilot pilot2 = new Pilot("Rubens Barrichello", 99);
                  db.Store(pilot2);
                  Console.WriteLine("Stored {0}", pilot2);
              }
          
              public static void RetrieveAllPilots(IObjectContainer db)
              {
                  IQuery query = db.Query();
                  query.Constrain(typeof(Pilot));
                  IObjectSet result = query.Execute();
                  ListResult(result);
              }
          
              public static void RetrievePilotByName(IObjectContainer db)
              {
                  IQuery query = db.Query();
                  query.Constrain(typeof(Pilot));
                  query.Descend("_name").Constrain("Michael Schumacher");
                  IObjectSet result = query.Execute();
                  ListResult(result);
              }
              
              public static void RetrievePilotByExactPoints(IObjectContainer db)
              {
                  IQuery query = db.Query();
                  query.Constrain(typeof(Pilot));
                  query.Descend("_points").Constrain(100);
                  IObjectSet result = query.Execute();
                  ListResult(result);
              }
          
              public static void RetrieveByNegation(IObjectContainer db)
              {
                  IQuery query = db.Query();
                  query.Constrain(typeof(Pilot));
                  query.Descend("_name").Constrain("Michael Schumacher").Not();
                  IObjectSet result = query.Execute();
                  ListResult(result);
              }
          
              public static void RetrieveByConjunction(IObjectContainer db)
              {
                  IQuery query = db.Query();
                  query.Constrain(typeof(Pilot));
                  IConstraint constr = query.Descend("_name")
                          .Constrain("Michael Schumacher");
                  query.Descend("_points")
                          .Constrain(99).And(constr);
                  IObjectSet result = query.Execute();
                  ListResult(result);
              }
          
              public static void RetrieveByDisjunction(IObjectContainer db)
              {
                  IQuery query = db.Query();
                  query.Constrain(typeof(Pilot));
                  IConstraint constr = query.Descend("_name")
                          .Constrain("Michael Schumacher");
                  query.Descend("_points")
                          .Constrain(99).Or(constr);
                  IObjectSet result = query.Execute();
                  ListResult(result);
              }
          
              public static void RetrieveByComparison(IObjectContainer db)
              {
                  IQuery query = db.Query();
                  query.Constrain(typeof(Pilot));
                  query.Descend("_points")
                          .Constrain(99).Greater();
                  IObjectSet result = query.Execute();
                  ListResult(result);
              }
          
              public static void RetrieveByDefaultFieldValue(IObjectContainer db)
              {
                  Pilot somebody = new Pilot("Somebody else", 0);
                  db.Store(somebody);
                  IQuery query = db.Query();
                  query.Constrain(typeof(Pilot));
                  query.Descend("_points").Constrain(0);
                  IObjectSet result = query.Execute();
                  ListResult(result);
                  db.Delete(somebody);
              }
              
              public static void RetrieveSorted(IObjectContainer db)
              {
                  IQuery query = db.Query();
                  query.Constrain(typeof(Pilot));
                  query.Descend("_name").OrderAscending();
                  IObjectSet result = query.Execute();
                  ListResult(result);
                  query.Descend("_name").OrderDescending();
                  result = query.Execute();
                  ListResult(result);
              }
          
              public static void ClearDatabase(IObjectContainer db)
              {
                  IObjectSet result = db.QueryByExample(typeof(Pilot));
                  foreach (object item in result)
                  {
                      db.Delete(item);
                  }
              }
          }
      }




      www.db4o.com