Collections

Resource collections are designed to be similar to how Django query sets work and provide a similar API.

ElementCollection

ElementCollections are available on all elements that inherit from smc.base.model.Element, and are also available for general searching across any element with an SMC entry point.

An ElementCollection can be constructed without making a single query to the SMC database. No query will occur until you do something to evaluate the collection.

You can evaluate a collection in the following ways:

  • Iteration. An ElementCollection is iterable, and it executes the SMC query the first time you iterate over it. For example, this will retrieve all host elements:

    >>> for host in Host.objects.all():
...    print(host.name, host.address)

  • list(). Force evaluation of a collection by calling list() on it:

    >>> elements = list(Host.objects.all())

  • first(). Helper collection method to retrieve only the first element in the search query:

    >>> host = Host.objects.iterator()
>>> host.first()
Host(name=SMC)

If you don’t need all results and only a single element, rather than getting an ElementCollection iterator, you can obtain this directly from the CollectionManager:

>>> Host.objects.first()
Host(name=SMC)

  • last(). Helper collection method to retrieve only the last element in the search query:

    >>> host = Host.objects.iterator()
>>> host.last()
Host(name=kali3)

  • exists(). Helper collection method to evaluate whether there are results:

    >>> hosts = Host.objects.filter('1.1.1.1')
>>> if hosts.exists():
...   for host in list(hosts):
...     print(host.name, host.address)
...
('hax0r', '1.1.1.1')
('host', '1.1.1.1')
('hostelement', '1.1.1.1')
('abcdefghijklmnop', '1.1.1.1')

  • count(). Helper collection method which returns the number of results. You can still obtain the results after:

    >>> it = Router.objects.iterator()
>>> query1 = it.filter('10.10.10.1')
>>> query1.count()
3
>>> list(query1)
[Router(name=Router-110.10.10.10), Router(name=Router-10.10.10.10), Router(name=Router-10.10.10.1)]

  • batch(). Iterator returning batches of results with specific by quantity. If limit() is also chained, it is ignored as batch and limit are mutually exclusive operations.

    >>> for hosts in Host.objects.batch(2):
...   print(hosts)
...
[Host(name=SMC), Host(name=172.18.1.135)]
[Host(name=172.18.2.254), Host(name=host)]
[Host(name=host-54.76.110.156), Host(name=host-192.168.4.135)]
[Host(name=external primary DNS resolver), Host(name=host-192.168.4.94)]
...

Methods that return a new ElementCollection

There are multiple methods in an ElementCollection that allow you to refine how the query or results are returned. Each chained method returns a new ElementCollection with aggregated search parameters.

  • filter(). Provide a filter string to narrow the search to a string value that will be used in a ‘contains’ match:

    >>> host = Host.objects.filter('172.18.1')
>>> list(host)
[Host(name=172.18.1.135), Host(name=SMC)]

filter can also take a keyword argument to filter specifically on an attribute. The keyword argument should match a valid attribute for the element type, and value to match:

>>> list(Router.objects.filter(address='10.10.10.1'))
[Router(name=Router-10.10.10.1)]

Note

Two additional keyword arguments can be passed to filter, exact_match=True and/or case_sensitive=False.

  • limit(). Limit the number of results to return.

    >>> list(Host.objects.all().limit(3))
[Host(name=SMC), Host(name=172.18.1.135), Host(name=172.18.2.254)]

  • all(). Return all results.

    >>> list(Host.objects.all())

Basic rules on searching

  • By default searches use a ‘contains’ logic. If you specify a filter string, the SMC API will return elements that contain that string. Therefore, if partial searches are performed, you may receive multiple matches:

    >>> list(Router.objects.filter('10.10'))
[Router(name=Router-110.10.10.10), Router(name=Router-10.10.10.10), Router(name=Router-10.10.10.1)]

  • When the search is evaluated, the elements returned contain only meta data and not the full payload for each element matching the search. The search query is built based on provided parameters to narrow the scope and only a single query is made to SMC.

  • When using a filter, the SMC API will search the name, comment and relevant field/s for the element type selected.

    Each element type will have it’s own searchable fields. For example, in addition to the name and comment field, a Host element will search the address and secondary address fields. This is automatic.

    For example, the following would find Host elements with this value in any of the Host fields specified above:

    >>> Host.objects.filter('111.111.111.111')

  • Setting exact_match=True on the filter query will only match on an element’s name or comment field and is a case sensitive match. The SMC is case sensitive, so unless you need an element by exact case, this field is not required. By default, exact_match=False.

  • In v0.5.6, case_sensitive=False can be set on the filter query to change the behavior of case sensitive matches. If not set, case_sensitive=True.

  • Using a keyword argument with ‘filter’ will provide element introspection against the attributes to perform an exact match. In general, using a kwarg is most effective when searching for network elements. Since the default search is a ‘contains’ match, a search for ‘10.10.10.1’ may return elements with values: ‘10.10.10.1’, ‘10.10.10.10’, and ‘110.10.10.1’. Using an attribute/value would override the default search behavior and attempt to only match on the specified attribute:

    >>> list(Router.objects.filter('10.10.10.1'))
[Router(name=Router-110.10.10.10), Router(name=Router-10.10.10.10), Router(name=Router-10.10.10.1)]

The above query returns multiple elements contains matches. To explicitly define the attribute to make an exact match, change the filter to use a kwarg (the address attribute is the defined ipaddress for smc.elements.network.Router):

>>> list(Router.objects.filter(address='10.10.10.1'))
[Router(name=Router-10.10.10.1)]

Note

When using keyword matching with filter, a single query will be performed using the attribute value, returning a list of ‘contains’ matches. For each element match returned from the first query, an additional query is performed to retrieve the element attributes.

To reduce the number of additional queries performed when using keyword matching, use a limit on the number of return elements:

>>> list(Router.objects.filter(address='10.10.10.1').limit(1))
[Router(name=Router-10.10.10.1)]

Additional Examples

Obtain an iterator from the collection manager for re-use:

>>> iterator = Router.objects.iterator()
>>> query1 = iterator.filter('10.10.10.1')
>>> list(query1)
[Router(name=Router-110.10.10.10), Router(name=Router-10.10.10.10), Router(name=Router-10.10.10.1)]
>>> query2 = query1.filter(address='10.10.10.1')
>>> list(query2)
[Router(name=Router-10.10.10.1)]

Access a collection directly on an Element type:

>>> list(Host.objects.all())
[Host(name=SMC), Host(name=172.18.1.135), Host(name=172.18.2.254), Host(name=host)]
...
>>> list(TCPService.objects.filter('HTTP'))
[TCPService(name=HTTPS_No_Decryption), TCPService(name=Squid HTTP proxy), TCPService(name=HTTP to Web SaaS)]

Limit number of return entries:

>>> list(Host.objects.limit(3))
[Host(name=SMC), Host(name=172.18.1.135), Host(name=172.18.2.254)]

Limit and filter the results using a chainable syntax:

>>> list(Host.objects.filter('172.18.1').limit(5))
[Host(name=172.18.1.135), Host(name=SMC), Host(name=TIE Server), Host(name=172.18.1.93)]

Get a host collection when partial IP address known:

>>> list(Host.objects.filter('192.168'))
[Host(name=aws-192.168.4.254), Host(name=host-192.168.4.135), Host(name=host-192.168.4.94), Host(name=host-192.168.4.79)]

When filtering is performed, by default search queries will ‘wildcard’ the results. To only return an exact match of the search query, use the optional flag ‘exact_match’:

>>> list(TCPService.objects.filter('8080'), exact_match=True))
[TCPService(name=TCP_8080), TCPService(name=HTTP proxy), TCPService(name=SSH), TCPService(name=SSM SSH)]

Additional convenience functions are provided on the collections to simplify navigating through results such as count, first, and last:

>>> query1 = iterator.filter('10.10.10.1')
>>> if query1.exists():
...   list(query1.all())
...
[Router(name=Router-110.10.10.10), Router(name=Router-10.10.10.10), Router(name=Router-10.10.10.1)]

>>> list(query1)
[Router(name=Router-110.10.10.10), Router(name=Router-10.10.10.10), Router(name=Router-10.10.10.1)]
>>> query1.first()
Router(name=Router-110.10.10.10)
>>> query1.last()
Router(name=Router-10.10.10.1)
>>> query1.count()
3
>>> query2 = query1.filter(address='10.10.10.1')  # Add kwarg to new query
>>> list(query2)
[Router(name=Router-10.10.10.1)]