2

I need to store IPv6 addresses in a Mongodb database using Rails3 + Mongoid.

There will also (mostly) be IPv4 addresses in the collection.

I need addresses to be stored as decimals since I have to query for addresses that belongs to a network (I'll store networks and addresses in distinct collections).

I've used BigDecimals to store those addresses (since IPv6 addresses are 128bit long) but when I try to find which addresses belongs to a network (concretely: between network and broadcast addresses), I don't find any working solution.

Mongoid "gte" and "lte" only seems to work on integers (BigDecimals are actually strings) and returns an empty list, and I don't find a way to query my mongoid model for a string range.

MongoDB seems to allow this (http://www.mongodb.org/display/DOCS/min+and+max+Query+Specifiers) but I don't find the corresponding syntax in mongoid doc.

Queries like the following raises awful "db assertion failure" :

Network.min(address: ip.network.to_i.to_s).max(address: ip.broadcast.to_i.to_s)

"ip.to_i.to_s" provides a string representation of the decimal address since I'm using IPAddress gem.

Same error with just 'to_i' or "BigDecimal.new(ip.network.to_i)"

The other solution is to store v6 addresses in 2 64bit integers but it's far more complex for range queries and I'd like to use the same behavior for v6 and v4 addresses.

Does anyone have experience on a clean way to deal with IPv6 addresses queries in a database ?

Here is my current Network model :

class Network

  # INCLUSIONS

    include Mongoid::Document
    include Mongoid::Timestamps

  # RELATIONS

    belongs_to :vlan

  # FIELDS

    field :description
    field :address, type: BigDecimal
    field :prefix,  type: Integer
    field :v6,      type: Boolean
    field :routed,  type: Boolean

  # VALIDATIONS

    validates :ip,
      presence: true

    # Address must be a valid IP address
    validate :ip do
      errors.add(:ip, :invalid) unless ip? && ip == ip.network
    end

  # INSTANCE METHODS

    # Returns string representation of the address
    def address
      ip.to_s if ip
    end

    def address= value
      raise NoMethodError, 'address can not be set directly'
    end

    # Provides the IPAddress object
    def ip
      unless @ip.is_a?(IPAddress) || self[:address].blank?
        # Generate IP address
        if self[:v6]
          @ip = IPAddress::IPv6.parse_u128 self[:address].to_i
        else
          @ip = IPAddress::IPv4.parse_u32 self[:address].to_i
        end
        # Set IP prefix
        @ip.prefix = self[:prefix] if self[:prefix]
      end
      @ip
    end

    # Sets network IP
    def ip= value
      value          = value.to_s
      @ip            = value
      self[:address] = nil
      self[:prefix]  = nil
      self[:v6]      = nil
      begin
        @ip            = IPAddress value
        self[:address] = @ip.to_i
        self[:prefix]  = @ip.prefix
        self[:v6]      = @ip.ipv6?
      rescue
      end
    end

    # Whether IP is a IPAddress object
    def ip?
      ip.is_a? IPAddress
    end

    # Provides network prefix
    def prefix
      return ip.prefix if ip?
      self[:prefix]
    end

    def prefix= value
      raise NoMethodError, 'prefix can not be set directly'
    end

    # Provides string representation of the network
    def to_s
      ip? ? ip.to_string : @ip.to_s
    end

    def subnets
      networks = Network.min(address: ip.network.to_i.to_s).max(address: ip.broadcast.to_i.to_s)
      return networks
    end

end

Subnets method is the one I'm working on, to detect networks nested into the current one.

Note that I'd like to avoid "strong" db relations between networks/subnets and upcoming hosts addresses to keep them dynamic.

Update:

Here's my final class that's working fine to manage nested IP networks.

Addresses are stored as fixed length strings in hexadecimal. They can be stored in base 32 to match real addresses size but hexa is better for readability.

Subnets method provides a list of all subnets of the current network.

class Network

  # INCLUSIONS

    include Mongoid::Document
    include Mongoid::Timestamps

  # RELATIONS

    belongs_to :vlan

  # FIELDS

    field :description
    field :address, type: String
    field :prefix,  type: Integer
    field :routed,  type: Boolean
    field :v6,      type: Boolean

  # VALIDATIONS

    validates :ip,
      presence: true

    # Address must be a valid IP address
    validate do
      errors.add(:ip, :invalid) unless ip? && ip == ip.network
    end

    validate do
        errors.add(:ip, :prefix_invalid_v6) if ip && ip.ipv6? && (self[:prefix] < 0 || self[:prefix] > 64)
    end

  # INSTANCE METHODS

    # Returns string representation of the address
    def address
      ip.to_s if ip
    end

    def address= value
      raise NoMethodError, 'address can not be set directly'
    end

    # Provides the IPAddress object
    def ip
      unless @ip.is_a?(IPAddress) || self[:address].blank?
        # Generate IP address
        if v6
          @ip = IPAddress::IPv6.parse_u128 self[:address].to_i(16)
        else
          @ip = IPAddress::IPv4.parse_u32 self[:address].to_i(16)
        end
        # Set IP prefix
        @ip.prefix = self[:prefix] if self[:prefix]
      end
      @ip
    end

    # Sets network IP
    def ip= value
      value          = value.to_s
      @ip            = value
      self[:address] = nil
      self[:prefix]  = nil
      self[:v6]      = nil
      begin
        @ip            = IPAddress value
        self[:address] = @ip.to_i.to_s(16).rjust((@ip.ipv4? ? 8 : 32), '0')
        self[:prefix]  = @ip.prefix
        self[:v6]      = @ip.ipv6?
      rescue
      end
    end

    # Whether IP is a IPAddress object
    def ip?
      ip.is_a? IPAddress
    end

    # Provides network prefix
    def prefix
      return ip.prefix if ip?
      self[:prefix]
    end

    def prefix= value
      raise NoMethodError, 'prefix can not be set directly'
    end

    # Provides string representation of the network
    def to_s
      ip? ? ip.to_string : @ip.to_s
    end

    # Provides nested subnets list
    def subnets
      length= ip.ipv4? ? 8 : 32
      networks = Network.where(
        v6: v6,
        :address.gte => (ip.network.to_i.to_s(16)).rjust(length, '0'),
        :address.lte => (ip.broadcast.to_i.to_s(16).rjust(length, '0')),
        :prefix.gte  => ip.prefix
      ).asc(:address).asc(:prefix)
    end

end 
  • I've just seen that Mongoid/MongoDB only seems to support signed 64bit integers...so it's even not possible to store the decimal address in 2 unsigned 64bit int...I could try with 4 32bit ints but this will be a mess... – Gauthier Delacroix Jul 13 '12 at 14:49

1 Answers1

2

A possible way to do it is to store the network in binary represented as string. And without including the meaningless trailing zeros (for a /12 you keep the first 12 bits).

Example: the network 192.168.1.0 is in reality 11000000101010000000000100000000. Let's evacuate the trailing zeros : 110000001010100000000001 as you can count, it's a /24.

You can find all its subnets :

Network.where(:address_bin => /^110000001010100000000001/, :v6 => false)

And that uses an index on :address_bin if any. :v6 => false is to match subnets of the same type.

Back to the source :)


Other way, simpler, your issue is for ipv6, but in ipv6, the last 64 bits of the address are reserved for the hosts in the subnet. For example, if you have a /40, you have only 64-40 = 24bits left to deals with subnets (if any).

So you can store the network address as a integer of 64bit, the first 64bits of the 128bit address ; the 64 last bits are forcedly zeros.

Maxime Garcia
  • 610
  • 3
  • 6
  • I tried the second way but mongoid/mongodb only stores SIGNED 64bit integers, so even the network part doesn't fit in an integer value. – Gauthier Delacroix Jul 16 '12 at 09:59
  • The first solution is dangerous since I can't be sure that all trailing zeroes have to be removed. Example : 192.168.0.0/24 => 16 trailing zeroes, only 8 to remove. – Gauthier Delacroix Jul 16 '12 at 10:03
  • Oh good catch on the signed issue, I missed it. But you can still use it adding 2^63 when retrieving or adding 2^63 before storing. It is very common, for example, storing an amount in cents as an integer. You can make it less pain in the ass creating a special type. See the "Custom field serialization" part of http://mongoid.org/en/mongoid/docs/documents.html#fields For the trailing zero, it was implicit, for a /12, you keep only the first 12 bits and so on. I edited to not confuse further readers. – Maxime Garcia Jul 16 '12 at 12:58
  • I even could store addresses in base 32 to match real address size, but hexadecimal has a better raw readability, at least for v6. Here, hexa addresses takes twice the real address size (8 bytes for v4, 32 bytes for v6). – Gauthier Delacroix Jul 16 '12 at 15:07