Extensions

smc-python provides additional extensions to extend the base library. Extensions are installed as separate packages and will have the dependency on the base smc-python library.

Available extensions:

  • smc-python-monitoring

smc-python-monitoring

smc-python-monitoring API provides a monitoring interface to the SMC to perform queries for dynamic engine components such as blacklists, connections, routes, vpn’s, users and logs.

Capabilities in the API implement the functionality found in the SMC Log Viewer and engine level monitoring.

Query

A Query is the top level object used to construct parameters to make queries to the SMC.

Query is the parent class for all monitors in package smc_monitoring.monitors

Each monitor type will have it’s own predefined set of log fields that are considered ‘default’ for the query type. These will correlate closely to the default fields you will see in the SMC when viewing the same information (Connections, VPN SAs, Blacklist, etc).

Each query also has a specific formatter which defines how the data is returned from the query. Formatters are defined in smc_monitoring.models.formats.

Each formatter type allows customization of the field_format and allows a value of ‘pretty’, ‘name’ or ‘id’. By default ‘pretty’ is used as the format which aligns with the column names in the SMC monitoring views.

class smc_monitoring.models.query.Query(definition=None, target=None, format=None, **sockopt)[source]

Query is the top level structure for controlling requests over the SMC websocket protocol. Any keyword arguments are passed through from inheriting classes are passed through as socket options for smc_monitoring.wsocket.SMCSocketProtocol.

Variables:
  • request (dict) – built request, eventually sent to socket
  • format (TextFormat) – format settings for query
add_and_filter(*values)[source]

Add a filter using “AND” logic. This filter is useful when requiring multiple matches to evaluate to true. For example, searching for a specific IP address in the src field and another in the dst field.

See also

smc_monitoring.models.filters.AndFilter for examples.

Parameters:values – optional constructor args for smc_monitoring.models.filters.AndFilter. Typically this is a list of InFilter expressions.
Type:list(QueryFilter)
Return type:AndFilter
add_defined_filter(*value)[source]

Add a DefinedFilter expression to the query. This filter will be considered true if the smc.monitoring.values.Value instance has a value.

See also

smc_monitoring.models.filters.DefinedFilter for examples.

Parameters:value (Value) – single value for the filter. Value is of type smc_monitoring.models.values.Value.
Type:list(QueryFilter)
Return type:DefinedFilter
add_in_filter(*values)[source]

Add a filter using “IN” logic. This is typically the primary filter that will be used to find a match and generally combines other filters to get more granular. An example of usage would be searching for an IP address (or addresses) in a specific log field. Or looking for an IP address in multiple log fields.

See also

smc_monitoring.models.filters.InFilter for examples.

Parameters:values – optional constructor args for smc_monitoring.models.filters.InFilter
Return type:InFilter
add_not_filter(*value)[source]

Add a filter using “NOT” logic. Typically this filter is used in conjunction with and AND or OR filters, but can be used by itself as well. This might be more useful as a standalone filter when displaying logs in real time and filtering out unwanted entry types.

See also

smc_monitoring.models.filters.NotFilter for examples.

Parameters:values – optional constructor args for smc_monitoring.models.filters.NotFilter. Typically this is a list of InFilter expressions.
Type:list(QueryFilter)
Return type:OrFilter
add_or_filter(*values)[source]

Add a filter using “OR” logic. This filter is useful when matching on one or more criteria. For example, searching for IP 1.1.1.1 and service TCP/443, or IP 1.1.1.10 and TCP/80. Either pair would produce a positive match.

See also

smc_monitoring.models.filters.OrFilter for examples.

Parameters:values – optional constructor args for smc_monitoring.models.filters.OrFilter. Typically this is a list of InFilter expressions.
Type:list(QueryFilter)
Return type:OrFilter
add_translated_filter()[source]

Add a translated filter to the query. A translated filter syntax uses the SMC expressions to build the filter. The simplest way to see the syntax is to create a filter in SMC under Logs view and right click->Show Expression.

Parameters:values – optional constructor args for smc_monitoring.models.filters.TranslatedFilter
Type:list(QueryFilter)
Return type:TranslatedFilter
execute()[source]

Execute the query with optional timeout. The response to the execute query is the raw payload received from the websocket and will contain multiple dict keys and values. It is more common to call query.fetch_XXX which will filter the return result based on the method. Each result set will have a max batch size of 200 records. This method will also continuously return results until terminated. To make a single bounded fetch, call fetch_batch() or fetch_raw().

Parameters:sock_timeout (int) – event loop interval
Returns:raw dict returned from query
Return type:dict(list)
fetch_as_element()[source]

Each inheriting class will override this method if supported.

fetch_batch(formatter=<class 'smc_monitoring.models.formatters.TableFormat'>, **kw)[source]

Fetch and return in the specified format. Output format is a formatter class in smc_monitoring.models.formatters. This fetch type will be a single shot fetch unless providing max_recv keyword with a value greater than the default of 1. Keyword arguments available are kw in fetch_raw().

Parameters:formatter – Formatter type for data representation. Any type in smc_monitoring.models.formatters.
Returns:generator returning data in specified format

Note

You can provide your own formatter class, see smc_monitoring.models.formatters for more info.

fetch_live(formatter=<class 'smc_monitoring.models.formatters.TableFormat'>)[source]

Fetch a live stream query. This is the equivalent of selecting the “Play” option for monitoring fields within the SMC UI. Data will be streamed back in real time.

Parameters:formatter – Formatter type for data representation. Any type in smc_monitoring.models.formatters.
Returns:generator yielding results in specified format
fetch_raw(**kw)[source]

Fetch the records for this query. This fetch type will return the results in raw dict format. It is possible to limit the number of receives on the socket that return results before exiting by providing max_recv.

This fetch should be used if you want to return only the result records returned from the query in raw dict format. Any other dict key/values from the raw query are ignored.

Parameters:max_recv (int) – max number of socket receive calls before returning from this query. If you want to wait longer for results before returning, increase max_iterations (default: 0)
Returns:list of query results
Return type:list(dict)
static resolve_field_ids(ids, **kw)[source]

Retrieve the log field details based on the LogField constant IDs. This provides a helper to view the fields representation when using different field_formats. Each query class has a default set of field IDs that can easily be looked up to examine their fields and different label options. For example:

Query.resolve_field_ids(ConnectionQuery.field_ids)
Parameters:ids (list) – list of log field IDs. Use LogField constants to simplify search.
Returns:raw dict representation of log fields
Return type:list(dict)
update_filter(filt)[source]

Update the query with a new filter.

Parameters:filt (smc_monitoring.models.filters.QueryFilter) – change query to use new filter
update_format(format)[source]

Update the format for this query.

Parameters:format (smc_monitoring.models.formats) – new format to use for this query

Models

The models package consists of the building blocks that make up a query.

Each module represents different class models that simplify adding things like filters, specifying values and formats.

Filters

Filters are used by queries to refine how results are returned.

QueryFilter is the top level ‘interface’ for all filter types. The filter attribute of a QueryFilter provides access to the compiled query string used to build the filter. Each QueryFilter also has an update_filter method that can be used to swap new filters in and out of an existing query.

Filters can be added to queries using the add_XXX methods of the query, or by building the filters and adding to the query using query.update_filter(). Filters can be swapped in and out of a query.

Examples:

Build a query to return all records of alert severity high or critical:

query = LogQuery(fetch_size=50)
query.add_in_filter(
    FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.HIGH, Alerts.CRITICAL)])

If you prefer building your filters individually, it is not required to call the add_XX_filter methods of the query. You can also insert filters by building the filter and calling the update_filter method on the query:

query = LogQuery(fetch_size=50)
query.update_filter(
    InFilter(FieldValue(LogField.SERVICE), [ServiceValue('UDP/53', 'TCP/80')])

You can also replace existing query filters with new filters to re-use the base level query parameters such as fetch_size, format style, time/date ranges, etc.

Replace the existing query filter with a different filter:

new_filter = InFilter(FieldValue(LogField.SERVICE), [ServiceValue('UDP/53', 'TCP/80')])
query.update_filter(new_filter)

Note

it is also possible to update a filter by calling query.add_XX_filter methods multiple times. Each time will replace an existing filter if it exists.

For example, calling add_XX_filter methods multiple times to refine filter results:

query = LogQuery(fetch_size=50)
query.add_in_filter(    # First filter query - look for alert severity high and critical
    FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.HIGH, Alerts.CRITICAL)])

query.add_and_filter([    # Change filter to AND filter for further granularity
    InFilter(FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.HIGH, Alerts.CRITICAL)]),
    InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')])])
class smc_monitoring.models.filters.AndFilter(*filters)[source]

Bases: smc_monitoring.models.filters.QueryFilter

An AND filter combines other filter types and requires that each filter matches. An AND filter is a collection of QueryFilter’s, typically IN or NOT filters that are AND’d together.

Example of fetching 50 records for sources matching ‘192.168.4.84’ and a service of ‘TCP/80’:

query = LogQuery(fetch_size=50)
query.add_and_filter([
    InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')]),
    InFilter(FieldValue(LogField.SERVICE), [ServiceValue('TCP/80')])])
Parameters:filters (list or tuple) – Any filter type in smc.monitoring.filters.
class smc_monitoring.models.filters.CILikeFilter[source]

Bases: smc_monitoring.models.filters.QueryFilter

A CILikeFilter is a case insensitive LIKE string match filter.

class smc_monitoring.models.filters.CSLikeFilter[source]

Bases: smc_monitoring.models.filters.QueryFilter

A CSLikeFilter is a case sensitive LIKE string match filter.

class smc_monitoring.models.filters.DefinedFilter(value=None)[source]

Bases: smc_monitoring.models.filters.QueryFilter

A Defined Filter applied to a query will only match if the value specified has a value in the audit record/s.

Show only records that have a defined Action (read as ‘match if action has a value’):

query = LogQuery(fetch_size=50)
query.add_defined_filter(FieldValue(LogField.ACTION))

DefinedFilter’s can be used in AND, OR or NOT filter queries as well. Fetch the most recent 50 records for source 192.168.4.84 that have an application defined:

query = LogQuery(fetch_size=50)
query.add_and_filter([
    DefinedFilter(FieldValue(LogField.IPSAPPID)),
    InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')])])
Parameters:values (Value) – single value type to require on filter
class smc_monitoring.models.filters.InFilter(left, right)[source]

Bases: smc_monitoring.models.filters.QueryFilter

InFilter’s are made up of two parts, a left and a right. An InFilter is considered a match if evaluation of the left part is equivalent to one of the elements of the right part. The left part of an InFilter is made up of a target of type smc.monitoring.values.Value. The right part is made up of a list of the same type.

Search the Source field for IP addresses 192.168.4.84 or 10.0.0.252:

query = LogQuery(fetch_size=50)
query.add_in_filter(
    FieldValue(LogField.SRC), [IPValue('192.168.4.84', '10.0.0.252')])

Reverse the logic and search for IP address 192.168.4.84 in source and dest log fields:

query = LogQuery(fetch_size=50)
query.add_in_filter(
    IPValue('192.168.4.84'), [FieldValue(LogField.SRC, LogField.DST)])

InFilter’s are one of the most common filters and are often added to AND, OR or NOT filters for more specific matching.

Parameters:
class smc_monitoring.models.filters.NotFilter(*filters)[source]

Bases: smc_monitoring.models.filters.QueryFilter

A NOT filter provides the ability to suppress auditing based on a specific filter. A NOT filter is typically added to an AND filter to remove unwanted entries from the response.

Use only a NOT filter to a query and to ignore DNS traffic:

query = LogQuery(fetch_size=50)
query.add_not_filter(
    [InFilter(FieldValue(LogField.SERVICE), [ServiceValue('UDP/53')])])

The above example by itself is not overly useful, however you can use NOT filters with AND filters to achieve a logic like “Find source IP 192.168.4.68 and not service UDP/53 or TCP/80”:

query = LogQuery(fetch_size=50)
not_dns = NotFilter(
    [InFilter(FieldValue(LogField.SERVICE), [ServiceValue('UDP/53', 'TCP/80')])])
by_ip = InFilter(
    FieldValue(LogField.SRC), [IPValue('172.18.1.20')])

query.add_and_filter([not_dns, by_ip])
Parameters:filters (list or tuple) – Any filter type in smc.monitoring.filters.
class smc_monitoring.models.filters.OrFilter(*filters)[source]

Bases: smc_monitoring.models.filters.QueryFilter

An OR filter matches if any of the combined filters match. An OR filter is a collection of QueryFilter’s, typically IN or NOT filters that are OR’d together.

Example of fetching 50 records for sources matching ‘192.168.4.84’ or a service of ‘TCP/80’:

query = LogQuery(fetch_size=50)
query.add_or_filter([
    InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')]),
    InFilter(FieldValue(LogField.SERVICE), [ServiceValue('TCP/80')])])
Parameters:filters (list or tuple) – Any filter type in smc.monitoring.filters.
class smc_monitoring.models.filters.TranslatedFilter[source]

Bases: smc_monitoring.models.filters.QueryFilter

Translated filters use the SMC internal name alias and builds expressions to make more complex queries.

Example of using built in filter methods:

query = LogQuery(fetch_size=50)
query.format.timezone('CST')
query.format.field_format('name')

translated_filter = query.add_translated_filter()
translated_filter.within_ipv4_network('$Dst', ['192.168.4.0/24'])
translated_filter.within_ipv4_range('$Src', ['1.1.1.1-192.168.1.254'])
translated_filter.exact_ipv4_match('$Src', ['172.18.1.152', '192.168.4.84'])
exact_ipv4_match(field, values)[source]

An exact IPv4 address match on relevant address fields.

Parameters:
  • field (str) – name of field to filter on. Taken from ‘Show Filter Expression’ within SMC.
  • values (list) – value/s to add. If more than a single value is provided, the query is modified to use UNION vs. ==
  • complex (bool) – A complex filter is one which requires AND’ing or OR’ing values. Set to return the filter before committing.
within_ipv4_network(field, values)[source]

This filter adds specified networks to a filter to check for inclusion.

Parameters:
  • field (str) – name of field to filter on. Taken from ‘Show Filter Expression’ within SMC.
  • values (list) – network definitions, in cidr format, i.e: 1.1.1.0/24.
within_ipv4_range(field, values)[source]

Add an IP range network filter for relevant address fields. Range (between) filters allow only one range be provided.

Parameters:
  • field (str) – name of field to filter on. Taken from ‘Show Filter Expression’ within SMC.
  • values (list) – IP range values. Values would be a list of IP’s separated by a ‘-’, i.e. [‘1.1.1.1-1.1.1.254’]

Values

Values are used to provide searchable input for filters. Each value format is specific to the data type added to the filter. For example, an IPValue specifies IP’s or network values that can be added to a filter from smc_monitoring.models.filters.

Each constructor can be initialized in the following ways:

Single value:

IPValue('1.1.1.1')

Multiple values:

IPValue('1.1.1.1', '2.2.2.2')

As a list of values:

i = ['1.1.1.1', '3.3.3.3']
IPValue(*i)

The value attribute of each Value stores the query string as a list that is absorbed by the filter.

class smc_monitoring.models.values.ConstantValue(*constants)[source]

Bases: smc_monitoring.models.values.Value

Constant values can be used for log field values. For example, specifying a filter by Action can be simplified by specifying the constant for the action value. Constant values are not used for log field names (use FieldValue instead).

Searching for all actions of discard and block:

query = LogQuery(fetch_size=50)
query.add_in_filter(
    FieldValue(LogField.ACTION), [ConstantValue(Actions.DISCARD, Actions.BLOCK)])
Parameters:constants (list or str) – constant values
class smc_monitoring.models.values.ElementValue(*elements)[source]

Bases: smc_monitoring.models.values.Value

Element Values are used when creating a filter for an element already defined in the SMC. The element can be referenced by it’s type.

Search for a host element ‘kali’ in the ‘source’ log field:

query = LogQuery(fetch_size=50)
query.add_in_filter(
    FieldValue(LogField.SRC), [ElementValue(Host('kali'))])
Parameters:elements (list or str) – element definitions

Note

Using elements expands the search to potentially include a broader range of data. For example, a host can have multiple IP addresses, both ipv4 and ipv6.

class smc_monitoring.models.values.FieldValue(*fields)[source]

Bases: smc_monitoring.models.values.Value

FieldValue specifies a log field filter by either constant ID or name. The field name field is the internal name representation for the SMC. To find a given field name, within SMC go to Log Viewer and drag a field into the filter window, right click and select “Show Filter Expression”.

Using field value as filter for InFilter type:

query = LogQuery(fetch_size=50)
query.add_in_filter(
    FieldValue(LogField.SRC), [IPValue('192.168.4.84')])
Parameters:fields (list or str) – fields definitions by name or int ID

Note

If using constant values, consult smc.monitoring.constants.LogField for valid attributes.

class smc_monitoring.models.values.IPValue(*addresses)[source]

Bases: smc_monitoring.models.values.Value

IP Values specify IP addresses used for searching.

Search for IP address in source and dest fields:

query = LogQuery(fetch_size=50)
query.add_in_filter(
    IPValue('192.168.4.84'), [FieldValue(LogField.SRC, LogField.DST)])
Parameters:addresses (list or str) – address definitions
class smc_monitoring.models.values.ServiceValue(*services)[source]

Bases: smc_monitoring.models.values.Value

Service Values allow searches on the service field. When specifying the service value, specify as <protocol/port>. For example, ‘TCP/80’, ‘UDP/53’. For ICMP, specify as ICMP/Type/Code (Code is optional).

Search for any services with TCP port 80 and UDP port 53:

query = LogQuery(fetch_size=50)
query.add_in_filter(
    FieldValue(LogField.SERVICE), [ServiceValue('TCP/80', 'UDP/53')])
Parameters:services (list or str) – service definitions
class smc_monitoring.models.values.StringValue(*values)[source]

Bases: smc_monitoring.models.values.Value

String value match. Note that string matching can only be done on log fields that are of type string (no type conversions are done on non-string types). String matches are also exact.

Find all audits accessing URL play.googleapis.com:

query = LogQuery(fetch_size=50)
query.add_in_filter(
    FieldValue(LogField.HTTPREQUESTHOST),[StringValue('play.googleapis.com')])
Parameters:value (list) – string to match
class smc_monitoring.models.values.Value(values)[source]

Bases: object

Value is the topmost parent for all value types.

Variables:value – stores value formatted into dict

Formats

Field formats represent a way to control the format of the returned data. By modifying a field format, you can control field level settings such as wther to resolve IP’s via DNS, how to display field names and values and which fields to return in the query.

Each log format will return a different view type The most common and default for all queries is the TextFormat using a ‘pretty’ field format which is what you will see from the column data and values if using the SMC Log Viewer.

Return only a specific set of fields by id’s:

query = LogQuery(fetch_size=5)
query.format.field_ids([
    LogField.TIMESTAMP, LogField.NODEID, LogField.SRC,
    LogField.DST, LogField.PROTOCOL, LogField.ACTION])

Return only a specific set of fields by name:

query = LogQuery(fetch_size=5)
query.format.field_names(['Src', 'Dst'])

Note

If both field_ids and field_names are provided, they will be merged.

class smc_monitoring.models.formats.CombinedFormat(**kw)[source]

Bases: object

CombinedFormat provides a way to specify different field resolvers based on field name or ID. Keyword arguments provided will define a unique key that represents the format object and value is the format object itself.

For example, using a combined filter to resolve the TIMESTMAP field in text format, but source and destination fields in detailed format:

text = TextFormat()
text.field_ids([LogField.TIMESTAMP])

detailed = DetailedFormat()
detailed.field_ids([LogField.SRC, LogField.DST])

combined = CombinedFormat(tformat=text, dformat=detailed)

query = LogQuery(fetch_size=1, format=combined)

After executing the query, the raw record results will be formatted as a list of dict’s, which each record having a dict key equal to keyword argument input provided:

[{'dformat': {'Src Addr': '10.0.0.1', 'Dst Addr': '224.0.0.1'},
  'tformat': {'Creation Time': '2017-08-05 14:12:44'}},
 {'dformat': {'Src Addr': '10.0.0.1', 'Dst Addr': '224.0.0.1'},
  'tformat': {'Creation Time': '2017-08-05 14:12:44'}}]
  ...

The results can then be parsed and used to provide custom views as necessary.

Parameters:kw – key word arguments should use an identifier key that will be present in the results, and a value which is a format object type in smc.monitoring.formats.
class smc_monitoring.models.formats.DetailedFormat(field_format='pretty', **kw)[source]

Bases: smc_monitoring.models.formats.TextFormat

Detailed format does not do a Log value conversion as the TextFormat would, however does provide a field map in the first payload with characteristics of the fields in the return data. This might be a useful format to obtain conversion ID’s for specific fields or debugging.

class smc_monitoring.models.formats.FormatFieldMixin[source]

Bases: object

Format field methods for modifying behavior of a query.

field_format(name)[source]

Specify how the field name are printed in the response.

Parameters:
field_ids(ids)[source]

Add filter to show only fields with given field ID’s. Field ID’s can be mapped to the LogField constants in smc_monitoring.models.constants.LogField

Note

Set the return display mode for the Log field name by using field_format(). The display value name will match the name of the LogField constant.

field_names(names)[source]

Show only fields with given name. The name is the internal SMC name for the log field. The simplest way to obtain the name for a log field is from the SMC Log Viewer. Use the Log Viewer filter window to drag a column filter and select “Show Filter Expression”.

..note:: The log field name is case sensitive and is typically
using camelcase notation.
class smc_monitoring.models.formats.RawFormat(field_format='pretty')[source]

Bases: smc_monitoring.models.formats.FormatFieldMixin

Raw format is an abbreviated version of the detailed format. Fewer fields are provided and resolution of field values is not done.

class smc_monitoring.models.formats.TextFormat(field_format='pretty', **kw)[source]

Bases: smc_monitoring.models.formats.FormatFieldMixin

Text format with ‘pretty’ field formatting uses the same display to what you would see from the native SMC Log Viewer.

Keyword arguments can optionally be provided to set ‘resolving’ fields during instance creation, or they can be set on the instance afterwards by calling set_resolving().

set_resolving(**kw)[source]

Certain log fields can be individually resolved. Use this method to set these fields. Valid keyword arguments:

Parameters:
  • timezone (str) – string value to set timezone for audits
  • time_show_zone (bool) – show the time zone in the audit.
  • time_show_millis (bool) – show timezone in milliseconds
  • keys (bool) – resolve log field keys
  • ip_elements (bool) – resolve IP’s to SMC elements
  • ip_dns (bool) – resolve IP addresses using DNS
  • ip_locations (bool) – resolve locations
timezone(tz)[source]

Set timezone on the audit records. Timezone can be in formats: ‘US/Eastern’, ‘PST’, ‘Europe/Helsinki’

See SMC Log Viewer settings for more examples.

Parameters:tz (str) – timezone, i.e. CST

Constants

Constants used within smc_monitoring.models.values.Value values to simplify referencing log viewer data.

class smc_monitoring.models.constants.Actions[source]

Rule Actions

ALLOW = 1

Allowed

BLOCK = 13

Block

DISCARD = 0

Discard

DISCARD_PASSIVE = 4

Silent discard

PERMIT = 11

Permit the connection

REFUSE = 2

Reset

TERMINATE = 9

Terminate

TERMINATE_FAILED = 10

Failed terminating connection

TERMINATE_PASSIVE = 8

Silent terminate

TERMINATE_RESET = 12

Reset the connection

class smc_monitoring.models.constants.Alerts[source]

Alert actions

CRITICAL = 10

Critical alert

HIGH = 5

High alert

INFO = 1

Info alert

LOW = 3

Low alert

class smc_monitoring.models.constants.DataType[source]

Query by type of logs. This identifies which log types you are interested in filtering by, i.e. Audit, FW Logs, Third_Party, etc. Equivalent to the Query dropdown in SMC Log Viewer

class smc_monitoring.models.constants.LogField[source]

Log field constants can be referenced when creating filters such as Field Values. i.e. FieldValue(LogField.SRC). Each constant name is identical to the value when using the field format type of ‘name’ (with exception that the constant names are in upper case).

ACCELAPSED = 104

Elapsed time of connection in seconds

ACCRXBYTES = 106

Number of bytes received during connection

ACCRXPACKETS = 139

Number of packets received during connection

ACCTXBYTES = 105

Number of bytes sent during connection

ACCTXPACKETS = 138

Number of packets sent during connection

ACK = 29

Acknowledged Alert

ACTION = 14

Connection action

ALERT = 25

Type of alert

ALERTCOUNT = 603

Alert count

ALERTERTRACE = 600

Alerter trace (events) information (datatype:4)

ALERTSEVERITY = 602

Severity of situation

ALERTSTATUS = 604

Alert status

ALLOWEDDATATAG = 482

Allowed data type tag

APPLICATION = 800

Application

APPLICATIONCOMBINATIONFLAGS = 54

Anomaly information of certain combination of network application and client application.

APPLICATIONDETAIL = 801

Application Detail

APPLICATIONUSAGE = 52

The type of the application that caused sending this event.

ASPAMEMAILMESSAGEID = 155

Email message-ID

ASPAMEMAILSCORE = 153

Email score value

ASPAMEMAILSUBJECT = 152

Email subject

ASPAMRECEIVEREMAIL = 151

Receiver email address

ASPAMSENDEREMAIL = 150

Sender email address

ASPAMSENDERMTA = 154

Sender Message Transfer Agent IP address

AUTHENTICATIONCOUNTER = 850

Authentication counters

AUTHMETHOD = 133

Authentication Method element

AUTHNAME = 108

User name of authorized user

AUTHRULEID = 107

The rule number of the rule that led to the log creation

BALANCINGPROBING = 397

BALANCING_PROBING

BALANCINGSELECTION = 392

BALANCING_SELECTION

BLACKLISTENTRYDESTINATIONIP = 120

Blacklist entry destination IP address

BLACKLISTENTRYDESTINATIONIPMASK = 121

Blacklist entry destination IP address mask

BLACKLISTENTRYDESTINATIONIPPREFIXLEN = 173

Blacklist entry destination IP address prefix length

BLACKLISTENTRYDESTINATIONPORT = 125

Blacklist entry destination port

BLACKLISTENTRYDESTINATIONPORTRANGE = 126

Blacklist entry destination port range end

BLACKLISTENTRYDURATION = 127

Blacklist entry duration

BLACKLISTENTRYID = 117

None

BLACKLISTENTRYPROTOCOL = 122

Blacklist entry IP protocol

BLACKLISTENTRYSOURCEIP = 118

Blacklist entry source IP address

BLACKLISTENTRYSOURCEIPMASK = 119

Blacklist entry source IP address mask

BLACKLISTENTRYSOURCEIPPREFIXLEN = 172

Blacklist entry source IP address prefix length

BLACKLISTENTRYSOURCEPORT = 123

Blacklist entry source port

BLACKLISTENTRYSOURCEPORTRANGE = 124

Blacklist entry source port range end

BLACKLISTER = 128

Blacklister

CIPHERALG = 536

Cipher algorithm

CLIENTIPADDRESS = 403

Address of client causing event

COMPID = 3

The identifier of the creator of the log entry.

CONNDIRECTION = 310

Connection direction

CONNECTEDMACADDR = 447

Connected MAC addresses

CONNECTIVITY = 306

Connectivity

CONNSTATUS = 309

Connection status

CONNTYPE = 308

Connection type

CONTAINEDDATATAG = 483

Contained data type tag

CONTROLCOMMANDID = 28

None

DATATAG = 481

Data type tag

DATATAGS = 485

Data tags concerning the record

DATATYPE = 34

Data type

DHCPLEASEEXPIRES = 528

DHCP_LEASE_EXPIRES

DHCPLEASEGW = 529

DHCP_LEASE_GW

DHCPLEASEIP = 530

DHCP_LEASE_IP

DHCPLEASENETMASK = 531

DHCP_LEASE_NETMASK

DHCPLEASEPREFIXLEN = 498

DHCP_LEASE_PREFIXLEN

DHCPLEASERECEIVED = 532

DHCP_LEASE_RECEIVED

DHCPLEASES = 527

DHCP_LEASES

DPD = 543

Dead Peer Detection

DPORT = 10

Connection destination protocol port

DSCPMARK = 130

DSCP Mark

DST = 8

Connection destination IP address

DSTADDRS = 20008

Destination addresses

DSTIF = 13

Destination interface of firewall

DSTIPRANGE = 526

Destination IP Range

DSTVLAN = 113

Destination VLAN

DSTZONE = 47

Connection destination interface zone

ELEMENTDOMAIN = 415

Administrative Domain of Associated Element

ENDPOINT = 504

Local VPN end point

ENTERPRISEOID = 493

Enterprise OID

EVENT = 6

Logged event

EVENTADDRESS = 705

Notification destination

EVENTINFO = 701

Description for event

EVENTLOGID = 702

Data Identifier of the alert

EVENTTIME = 700

Time stamp of the alert

EVENTTYPE = 703

Type of event

EVENTUSER = 704

User who executed the action

EXPIRATIONTIME = 534

VPN SA expiration time

FACILITY = 22

Firewall subsystem

FILETYPECOMPAT = 56

The type of the file that caused sending this event.

FLAG = 114

None

FPCACHED = 57

Fingerprint match came from fingerprinting cache.

FW100INTERFACE = 431

FW100 Interface

FW100TRAFFICCOUNTERS = 430

Fw100 Traffic counters

FWACCEPTEDBYTES = 326

FW_ACCEPTED_BYTES

FWACCEPTEDPACKETS = 327

FW_ACCEPTED_PACKETS

FWACCOUNTEDBYTES = 336

FW_ACCOUNTED_BYTES

FWACCOUNTEDPACKETS = 337

FW_ACCOUNTED_PACKETS

FWADSLRXBYTES = 417

FW_ADSL_RX_BYTES

FWADSLTXBYTES = 416

FW_ADSL_TX_BYTES

FWDECRYPTEDBYTES = 332

FW_DECRYPTED_BYTES

FWDECRYPTEDPACKETS = 333

FW_DECRYPTED_PACKETS

FWDROPPEDBYTES = 328

FW_DROPPED_BYTES

FWDROPPEDPACKETS = 329

FW_DROPPED_PACKETS

FWENCRYPTEDBYTES = 330

FW_ENCRYPTED_BYTES

FWENCRYPTEDPACKETS = 331

FW_ENCRYPTED_PACKETS

FWFORWARDEDBYTES = 419

FW_FORWARDED_BYTES

FWFORWARDEDPACKETS = 418

FW_FORWARDED_PACKETS

FWINTERFACEKEY = 340

FW_INTERFACE_KEY

FWNATTEDBYTES = 334

FW_NATTED_BYTES

FWNATTEDPACKETS = 335

FW_NATTED_PACKETS

FWRECEIVEDBYTES = 322

FW_RECEIVED_BYTES

FWRECEIVEDPACKETS = 323

FW_RECEIVED_PACKETS

FWSENTBYTES = 324

FW_SENT_BYTES

FWSENTPACKETS = 325

FW_SENT_PACKETS

FWTRAFFIC = 342

FW Traffic

FWTRAFFICACCOUNTEDBYTES = 352

Accounted Bytes

FWTRAFFICACCOUNTEDPACKETS = 346

Accounted Packets

FWTRAFFICALLOWEDBYTES = 349

Allowed Bytes

FWTRAFFICALLOWEDPACKETS = 343

Allowed Packets

FWTRAFFICDISCARDEDBYTES = 350

Discarded Bytes

FWTRAFFICDISCARDEDPACKETS = 344

Discarded Packets

FWTRAFFICENCRYPTEDBYTES = 354

Encrypted Bytes

FWTRAFFICENCRYPTEDPACKETS = 348

Encrypted Packets

FWTRAFFICLOGGEDBYTES = 351

Logged Bytes

FWTRAFFICLOGGEDPACKETS = 345

Logged Packets

FWTRAFFICNATTEDBYTES = 353

Natted Bytes

FWTRAFFICNATTEDPACKETS = 347

Natted Packets

GENERICTRAPTYPE = 494

Generic Trap Type

HASHALG = 538

Hash Algorithm

HITS = 48

HITS

HTTPREQUESTHOST = 1586

HTTP request host

ICMPCODE = 101

ICMP code attribute

ICMPID = 102

ICMP identifier

ICMPTYPE = 100

ICMP type attribute

IKEDHGROUP = 901

Diffie-Hellman Group

IKELOCALID = 540

Local IKE ID

IKEREMOTEID = 541

Remote IKE ID

IKEV1MODE = 542

IKEv1 negotiation mode

INCIDENTCASE = 411

Incident Case

INFOMSG = 19

Information Message

INTERFACE = 35

Interface

IPCOMPRESSION = 546

IP Compression

IPSAPPID = 134

Network application detected in the connection

IPSECSSPI = 103

Inbound IPsec SPI value (hexadecimal)

LOGID = 2

Data Identifier

LOGIFTOPDESTINATIONIPADDRS = 446

Amount of traffic flowing to the most used destination IP addresses per logical interface

LOGIFTOPSOURCEIPADDRS = 445

Amount of traffic originating from the most used source IP addresses per logical interface

LOGIFTOPTCPDESTINATIONPORTS = 443

Amount of traffic on the most used TCP destination ports per logical interface

LOGIFTOPUDPDESTINATIONPORTS = 444

Amount of traffic on the most used UDP destination ports per logical interface

LOGSEVERITY = 805

Severity

LONGMSG = 601

Long field description of alert

MACALG = 537

MAC Algorithm

MESSAGEID = 804

Message Id

NATBALANCEID = 393

NAT_BALANCE_ID

NATDPORT = 18

Translated packet destination port

NATDST = 16

Translated packet destination IP address

NATMAPID = 394

NAT_MAP_ID

NATRULEID = 21

The rule number of the rule that led to the log creation

NATSPORT = 17

Translated packet source protocol port

NATSRC = 15

Translated packet source IP address

NATT = 544

NAT Traversal

NEGOTIATIONROLE = 539

SA Negotiation Role

NODECAPACITY = 321

Capacity

NODECONFIGURATION = 304

Current configuration

NODECONFIGURATIONTIMESTAMP = 305

Configuration upload time

NODEDYNUP = 303

Update package level

NODEHWSTATUS = 315

Node hardware status

NODEID = 4

Firewall or server node that passes this information

NODELOAD = 320

Node load

NODESTATUS = 300

Node status

NODEVERSION = 301

Node version

NONCONTAINEDDATATAG = 484

Non-contained data type tag

NUMALERTRESPONSES = 365

Number of alert responses performed by this engine

NUMBLACKLISTRESPONSES = 369

Number of blacklist responses performed by this engine

NUMBYTESRECEIVED = 12201

Number of bytes received, used for VPN

NUMBYTESSENT = 12200

Number of bytes sent, used for VPN

NUMDISCARDRESPONSES = 368

Number of discard responses performed by this engine

NUMLOGEVENTS = 363

Number of log events

NUMLOGRESPONSES = 364

Number of log responses performed by this engine

NUMPACKETSRECEIVED = 549

Number of packets received

NUMPACKETSSENT = 548

Number of packets sent

NUMRECORDRESPONSES = 366

Number of record responses performed by this engine

NUMRESETRESPONSES = 367

Number of reset responses performed by this engine

OBJECTDN = 410

User and Group Information

OBJECTID = 406

Special field for filtering Audit entries using the defined resources. Not present in the audit entries as such.

OBJECTKEY = 409

Element Id

OBJECTNAME = 407

Elements being manipulated

OBJECTTYPE = 408

Element Type

ORIGINNAME = 400

Name of component producing event

OUTBOUNDSPI = 533

Outbound IPsec SPI value (hexadecimal)

PASSEDBYTES = 388

PASSED_BYTES

PEERCOMPONENTID = 307

Peer component id

PEERENDPOINT = 506

Peer VPN end point

PEERSECURITYGATEWAY = 505

Peer VPN gateway

PFSDHGROUP = 547

PFS Diffie-Hellman Group

PHASE1FAIL = 511

IKE_PHASE1_FAIL

PHASE1SUCC = 510

IKE_PHASE1_SUCC

PHASE2FAIL = 513

IKE_PHASE2_FAIL

PHASE2SUCC = 512

IKE_PHASE2_SUCC

POTENTIALLYDUPLICATERESPONSE = 170

Potentially duplicate correlation response

PROBEFAIL = 500

PROBE_FAIL

PROBEOK = 399

PROBE_OK

PROTOCOL = 11

IP protocol

QOSCLASS = 129

QoS Class

QOSPRIORITY = 131

QoS Priority

RADIUSACCOUNTINGTYPE = 851

Radius Accounting Type

RECEIVEDLOGEVENTS = 361

RECEIVED_LOG_EVENTS

RECEPTIONTIME = 24

Reception Time on the log Server

RESOURCE = 806

Resource

RESULT = 405

Result state

RETSRCIF = 49

Return source interface of the connection

ROUTEBGPPATH = 167

Active BGP path

ROUTEDISTANCE = 162

Relative distance for route validation

ROUTEGATEWAY = 164

IP address of the gateway for the route

ROUTEMETRIC = 163

Protocol specific metric value

ROUTENETMASK = 161

Netmask address of the network

ROUTENETWORK = 160

Network address of the network

ROUTEOSPFLSATYPE = 166

Type of OSPF LSA’s

ROUTETYPE = 165

Type of route

RTT = 109

Round trip time of connection establishing

RULECOUNTERS = 412

RULE_COUNTERS

RULEHITS = 413

RULE_HITS

RULEID = 20

Rule tag value of acceptance rule

RWPHTTPREFERRER = 832

HTTP Referrer

RWPHTTPUSERAGENT = 830

HTTP User Agent

RWPSERVICENAME = 831

SSL VPN Portal Service Name

SAAUTHALG = 520

SA_AUTH_ALG

SABUNDLE = 514

SA_BUNDLE

SACIPHERALG = 518

SA_CIPHER_ALG

SACLASS = 535

SA Type

SACOMPRESSIONALG = 519

SA_COMPRESSION_ALG

SAEXPIREHARDLIMIT = 524

SA_EXPIRE_HARDLIMIT

SAEXPIRESOFTLIMIT = 523

SA_EXPIRE_SOFTLIMIT

SAINCOMING = 517

SA_INCOMING

SAKBHARDLIMIT = 522

SA_KB_HARDLIMIT

SAKBSOFTLIMIT = 521

SA_KB_SOFTLIMIT

SARESPONDER = 516

SA_RESPONDER

SATYPE = 515

SA_TYPE

SECURITYGATEWAY = 502

VPN gateway

SELECTEDCACHE = 396

SELECTED_CACHE

SELECTEDRTT = 395

SELECTED_RTT

SENDER = 5

None

SENDERDOMAIN = 38

Administrative Domain of Event Sender

SENDERTYPE = 31

Sender type

SENSORALLOWEDINSPECTEDTCPCONNECTIONS = 437
SENSORALLOWEDINSPECTEDUDPCONNECTIONS = 438
SENSORALLOWEDUNINSPECTEDTCPCONNECTIONS = 439
SENSORALLOWEDUNINSPECTEDUDPCONNECTIONS = 440
SENSORDISCARDEDTCPCONNECTIONS = 441
SENSORDISCARDEDUDPCONNECTIONS = 442
SENSORINSPECTEDBYTES = 357

Bytes inspected by sensor

SENSORINSPECTEDPACKETS = 358

Packets inspected by sensor

SENSORINTERFACEKEY = 370

Sensor interface key

SENSORLOSTBYTES = 359

Bytes lost in sensor

SENSORLOSTPACKETS = 360

Packets lost in sensor

SENSORPROCESSEDBYTES = 355

Bytes processed by sensor

SENSORPROCESSEDPACKETS = 356

Packets processed by sensor

SENSORRECEIVEDBYTES = 338

Bytes received by sensor

SENSORRECEIVEDPACKETS = 339

Packets received by sensor

SENSORTRAFFIC = 372

Sensor traffic

SENSORTRAFFICCLOSEDTCPCONNECTIONS = 383

Closed TCP Connections

SENSORTRAFFICINSPECTEDPACKETS = 376

Inspected Packets

SENSORTRAFFICLOSTPACKETS = 375

Lost Packets

SENSORTRAFFICNEWTCPCONNECTIONS = 381

New TCP Connections

SENSORTRAFFICNUMBEROFALERTS = 380

Number of Alerts

SENSORTRAFFICOKCONNECTIONS = 378

OK Connections

SENSORTRAFFICPROCESSEDBYTES = 374

Processed Bytes

SENSORTRAFFICPROCESSEDPACKETS = 373

Processed Packets

SENSORTRAFFICSTATSOFPACKETS = 377

Stats Of Packets

SENSORTRAFFICSUSPICIOUSCONNECTIONS = 379

Suspicious Connections

SENSORTRAFFICTCPHANDSHAKES = 382

TCP Handshakes

SENSORTRAFFICTCPTIMEOUTS = 384

TCP Timeouts

SENTLOGEVENTS = 362

SENT_LOG_EVENTS

SERVICE = 27

Special field for filtering logs using the defined services. Not present in the log entries as such.

SERVICEKEY = 132

Service primary key, used in service resolving

SESSIONDOMAIN = 414

Administrative Domain of Login Session

SESSIONEVENT = 302

Session monitoring event code (1 = new, 2 = update, 3 = remove, 4 = all sessions sent)

SESSIONID = 802

Id of the User Session

SFPINGRESS = 900

SFP_INGRESS

SHAPINGCLASS = 386

SHAPING_CLASS

SHAPINGGUARANTEE = 389

SHAPING_GUARANTEE

SHAPINGLIMIT = 390

SHAPING_LIMIT

SHAPINGPRIORITY = 391

SHAPING_PRIORITY

SITCATEGORY = 37

The type of the situation that caused sending this event.

SITUATION = 1000

The identifier of the situation that caused sending this event.

SNMPRETSRCIF = 51

SNMP index of return source interface

SNMPSRCIF = 50

SNMP index of source interface

SNMPTRAPMAP = 490

SNMP Trap

SNMPTRAPOID = 491

SNMP Trap OID

SNMPTRAPVALUE = 492

SNMP Trap Value

SPORT = 9

Connection source protocol port

SRC = 7

Connection source IP address

SRCADDRESS = 398

SRC_ADDRESS

SRCADDRS = 20007

Source addresses

SRCIF = 12

Source interface of firewall

SRCIPRANGE = 525

Source IP Range

SRCVLAN = 112

Source VLAN

SRCZONE = 46

Connection source interface zone

SRVHELPERID = 110

Protocol agent identification

SSLVPNSESSIONMONID = 811

Id of the User Session

SSLVPNSESSIONMONRECEIVED = 809

Node’s local time when the SSL VPN session was created

SSLVPNSESSIONMONTIMEOUT = 810

Node’s local time when the SSL VPN session will time-out

SSLVPNSESSIONTYPETYPE = 808

SSL VPN session client type

STATE = 116

Connection state in connection monitoring

STATUSTYPE = 311

Status type

STORAGESERVERID = 30

Storage Server

SYSLOGTYPE = 111

Syslog message type

TAGINFO = 480

Type tags

TCPDUMPSTATUS = 318

TCPDump Monitoring Status

TCPENCAPSULATION = 545

TCP Encapsulation

TIMEOUT = 115

Connection timeout in connection monitoring

TIMESTAMP = 1

Time of creating the event record.

TLSALERTDESCRIPTION = 45

TLS/SSL Alert Message Description

TLSALERTLEVEL = 44

TLS/SSL Alert Message Alert Level

TLSCERTIFICATEVERIFYERRORCODE = 39

TLS/SSL Certificate verify error code

TLSCIPHERSUITE = 42

TLS/SSL cipher suite

TLSCOMPRESSIONMETHOD = 43

TLS/SSL compression method

TLSDECRYPTED = 137

The connection was decrypted and re-encrypted in the engine to perform deep inspection or application identification.

TLSDETECTED = 136

The connection uses SSL/TLS protocol.

TLSDOMAIN = 40

Domain name field in SSL/TLS certificate

TLSMATCH = 135

TLS Match detected in the connection. Note that a single connection can have any number of distinct TLS Matches.

TLSPROTOCOLVERSION = 41

TLS/SSL protocol version

TOTALBYTES = 387

TOTAL_BYTES

TPACCEPTEDBYTES = 465

TP_ACCEPTED_BYTES

TPACCEPTEDPACKETS = 466

TP_ACCEPTED_PACKETS

TPDROPPEDBYTES = 467

TP_DROPPED_BYTES

TPDROPPEDPACKETS = 468

TP_DROPPED_PACKETS

TPMEMUSAGE = 470

Third party memory usage

TPNODELOAD = 469

Third party device load

TPRECEIVEDBYTES = 461

TP_RECEIVED_BYTES

TPRECEIVEDPACKETS = 462

TP_RECEIVED_PACKETS

TPSENTBYTES = 463

TP_SENT_BYTES

TPSENTPACKETS = 464

TP_SENT_PACKETS

TPTRAFFICCOUNTERS = 460

Third party traffic counters

TRAFFICCOUNTERS = 319

Traffic counters

TRAFFICSHAPING = 385

TRAFFIC_SHAPING

TRANSIENT = 26

None

TUNNELINGLEVEL = 95

Number of tunneling protocol layers encapsulating this protocol layer

TYPE = 23

Log event severity type

TYPEDESCRIPTION = 404

Description of the event

URLCATEGORYGROUP = 53

The type of the URL that caused sending this event.

URLCATEGORYRISK = 55

The risk of the URL that caused sending this event.

USERNAME = 3001

Username if present

USERORIGINATOR = 401

Administrator causing event

USERROLE = 402

Roles of Administrator causing event

VPNBYTESRECEIVED = 509

VPN_BYTES_RECEIVED

VPNBYTESSENT = 508

VPN_BYTES_SENT

VPNID = 501

Desination VPN

VPNSRCID = 499

Source VPN

VPNSTATISTICS = 507

VPN_STATISTICS

VPNSTATUS = 503

VPN_STATUS

VPNTYPE = 611

VPN_TYPE

VULNERABILITYREFERENCES = 20000

Generated from situation and original situation.

WIRELESSCHANNEL = 448

Wireless Access Point’s channel

WIRELESSCONNECTIONS = 436

Number of wireless connections

WIRELESSMONITORING = 432

Wireless Monitoring

WIRELESSSECURITY = 435

Wireless Security mode

WIRELESSSSID = 433

Wireless SSID

WIRELESSSTATUS = 434

Wireless Status

ZIPEXPORTFILE = 420

Snapshot of element being manipulated

Formatters

Custom formats used to return data in different formats. These are used from the query itself when calling the fetch_as_format() method. For example, returning a LogQuery as a table:

query = LogQuery(fetch_size=200)
for log in query.fetch_batch(): # Default is TableFormat
    print(log)

As CSV:

query = LogQuery(fetch_size=200)
for log in query.fetch_batch(CSVFormat):
    print(log)

Each format also allows the ability to customize the fields that should be in the output. By default, each query type in smc_monitoring.monitors will have a class attribute field_ids which specify the default fields. These can be customized by modifying the query.format.field_ids([….]) parameter.

For example, modifying a routing query to return only destination interface and the route network:

query = RoutingQuery('sg_vm')
query.format.field_ids([LogField.DSTIF, LogField.ROUTENETWORK])
for log in query.fetch_batch():
    ...

The same field_id customization applies to all query types.

A simple way to view results is to use a RawDictFormat:

query = LogQuery(fetch_size=3)
query.format.field_names(['Src', 'Dst'])
for record in query.fetch_batch(RawDictFormat):
    ...

It is also possible to provide your own formatter. At a minimum you must provide a method called formatted in your class. The custom class should extend _Header to support custom field_ids within the query.

Note

Constants are defined in smc_monitoring.models.constants. Although there are many field values, not all field values will return results for every query. It is sometimes useful to log in to the SMC to verify available fields.

class smc_monitoring.models.formatters.CSVFormat(query)[source]

Bases: smc_monitoring.models.formatters._Header

Return the results in CSV format. The first line will be a comma separated string with the field header. This is an iterable that will return results in batches of 200 (max) per iteration.

exception smc_monitoring.models.formatters.InvalidFieldFormat[source]

Bases: exceptions.Exception

If using a complex format type such as combined, formatters are not supported. These specialized formats must be returned in raw dict format as they’ve been customized to return the data in a specific way.

class smc_monitoring.models.formatters.RawDictFormat(query)[source]

Bases: object

Return the data as a list in raw dict format. The results are not filtered with exception of the returned fields based on field_id filters. This is a convenience format for consistency, although you can also call the smc_monitoring.models.query.Query.fetch_raw method to get the same data.

class smc_monitoring.models.formatters.TableFormat(query)[source]

Bases: smc_monitoring.models.formatters._Header

Return the data in a table format. The field_id values will be used for the table header. Spacing will be calculated for each batch of results to align the table. The base spacing is determined by the header width, but adjusted wider if the data returned is wider. Anytime there is an adjustment to the width, a new table header will also be printed to visually realign. The query will return a max of 200 batch results per iteration.

Note

Table alignment will likely not be exact between batches as width is calculated per batch.

TimeRanges

Time formats are optionally used in a LogQuery to specify custom ranges for which to search ‘stored’ log events.

When adding a time format to a query, the start_time and end_time values need to be in milliseconds. The engine logs are stored in UTC time but in order to display the client side dates properly, you should set a timezone on the query.

There are helper methods to simplify retrieving for last_XXX period of time as well as custom range formats.

Set up a query with a time format:

query = LogQuery(fetch_size=50)
query.format.timezone('Europe/Helsinki')
query.time_range.last_five_minutes()

See also

custom_range() for more examples on creating custom time range formats.

class smc_monitoring.models.calendar.TimeFormat(start_ms=0, end_ms=0)[source]

Bases: object

Construct a time format to control the start and end times for a query. If unspecified, results will be limited by the fetch size quantity only. Helper methods are provided to simplify adding time based filters once the instance is constructed.

Parameters:
  • start_ms (int) – datetime object in milliseconds. Where to start the query in time. If your search should go backwards in time, specify the oldest time/date in start_time.
  • end_ms (int) – datetime object in milliseconds. Where to end the query in time.
custom_range(start_time, end_time=None)[source]

Provide a custom range for the search query. Start time and end time are expected to be naive datetime objects converted to milliseconds. When submitting the query, it is strongly recommended to set the timezone matching the local client making the query.

Example of finding all records on 9/2/2017 from 06:25:30 to 06:26:30 in the local time zone CST:

dt_start = datetime(2017, 9, 2, 6, 25, 30, 0)
dt_end = datetime(2017, 9, 2, 6, 26, 30, 0)

query = LogQuery()
query.format.timezone('CST')
query.time_range.custom_range(
    datetime_to_ms(dt_start),
    datetime_to_ms(dt_end))

for record in query.fetch_batch():
    print(record)

Last two minutes from current (py2):

now = datetime.now()
start_time = int((now - timedelta(minutes=2)).strftime('%s'))*1000

Specific start time (py2):

p2time = datetime.strptime("1.8.2017 08:26:42,76", "%d.%m.%Y %H:%M:%S,%f").strftime('%s')
p2time = int(s)*1000

Specific start time (py3):

p3time = datetime.strptime("1.8.2017 08:40:42,76", "%d.%m.%Y %H:%M:%S,%f")
p3time.timestamp() * 1000
Parameters:
  • start_time (int) – search start time in milliseconds. Start time represents the oldest timestamp.
  • end_time (int) – search end time in milliseconds. End time represents the newest timestamp.
end_time

Return the end time in datetime format. Will return 0 if end time is not specified.

Return type:datetime
last_day()[source]

Add time filter from current time back 1 day

last_fifteen_minutes()[source]

Add time from current time back 15 minutes

last_five_minutes()[source]

Add time from current time back 5 minutes

last_hour()[source]

Add time from current time back 1 hour

last_thirty_minutes()[source]

Add time from current time back 30 minutes

last_week()[source]

Add time filter from current time back 7 days.

start_time

Return the start time in datetime format. Will return 0 if start time is not specified.

Return type:datetime
smc_monitoring.models.calendar.datetime_from_ms(ms)[source]

Convenience to return datetime from milliseconds

Returns:datetime from ms
Return type:datetime
smc_monitoring.models.calendar.datetime_to_ms(dt)[source]

Convert an unaware datetime object to milliseconds. This datetime should be the time you would expect to see on the client side. The SMC will do the timestamp conversion based on the query timezone.

Returns:value representing the datetime in milliseconds
Return type:int
smc_monitoring.models.calendar.subtract_from_now(td)[source]

Subtract timedelta from current time

Monitors

The monitors package provides modules that represent individual monitoring areas within the SMC monitoring API. Each monitor type extends smc_monitoring.models.query.Query to provide a consistent API for adding filters and executing queries.

Blacklist

Blacklist Query provides the ability to view current blacklist entries in the SMC by target. Target is defined as the cluster or engine. Retrieved results will have a reference to the entry and hence be possible to remove the entry.

query = BlacklistQuery('sg_vm')
query.format.timezone('CST')

Optionally add an “InFilter” to restrict search to a specific field:

query.add_in_filter(
    FieldValue(LogField.BLACKLISTENTRYSOURCEIP), [IPValue('2.2.2.2')])

An InFilter can also use a network based syntax:

query.add_in_filter(
    FieldValue(LogField.BLACKLISTENTRYSOURCEIP), [IPValue('2.2.2.0/24')])

Or combine filters using “AndFilter” or “OrFilter”. Find an entry with source IP 2.2.2.2 OR 2.2.2.5:

ip1 = InFilter(FieldValue(LogField.BLACKLISTENTRYSOURCEIP), [IPValue('2.2.2.2')])
ip2 = InFilter(FieldValue(LogField.BLACKLISTENTRYSOURCEIP), [IPValue('2.2.2.5')])
query.add_or_filter([in_filter, or_filter])

Get the results of the query in the default TableFormat:

for entry in query.fetch_batch():
    print(entry)

Delete any blacklist entries with a source IP within a network range of 3.3.3.0/24:

query = BlacklistQuery('sg_vm')
query.add_in_filter(
    FieldValue(LogField.BLACKLISTENTRYSOURCEIP), [IPValue('3.3.3.0/24')])

for record in query.fetch_as_element():    # <-- must get as element to obtain delete() method
    record.delete()

See also

smc_monitoring.models.filters for more information on creating filters

class smc_monitoring.monitors.blacklist.BlacklistEntry(**kw)[source]

Bases: object

A blacklist entry represents an entry in the engines kernel table indicating that a source/destination/port/protocol mapping is currently being blocked by the engine. To remove a blacklist entry from an engine, retrieve all entries as element and remove the entry of interest by called delete on the element.

The simplest way to use search filters with a blacklist entry is to examine the BlacklistQuery field_ids and use these constant fields as InFilter definitions on the query.

blacklist_id

Blacklist entry ID. Useful if you want to locate the entry within the SMC UI.

Return type:str
delete()[source]

Delete the entry from the engine where the entry is applied.

Raises:DeleteElementFailed
Returns:None
dest_ports

Destination ports for this blacklist entry. If no ports are specified, ‘ANY’ is returned.

Return type:str
destination

Destination network/netmask for this blacklist entry.

Return type:str
duration

Duration for the blacklist entry.

Return type:int
engine

The engine for this blacklist entry.

Return type:str
href

The href for this blacklist entry. This is the reference to the entry for deleting the entry.

Return type:str
protocol

Specified protocol for the blacklist entry. If none is specified, ‘ANY’ is returned.

Return type:str
source

Source address/netmask for this blacklist entry.

Return type:str
source_ports

Source ports for this blacklist entry. If no ports are specified (i.e. ALL ports), ‘ANY’ is returned.

Return type:str
timestamp

Timestamp when this blacklist entry was added.

Return type:str
class smc_monitoring.monitors.blacklist.BlacklistQuery(target, timezone=None, **kw)[source]

Bases: smc_monitoring.models.query.Query

Query existing blacklist entries for a given cluster/engine. It is generally recommended to set your local timezone when making a query to convert the timestamp into a relevant format.

Parameters:
  • target (str) – NAME of the engine or cluster
  • timezone (str) – timezone for timestamps.

Note

Timezone can be in the following formats: ‘US/Eastern’, ‘PST’, ‘Europe/Helsinki’. More example time zone formats are available in the SMC Log Viewer -> Settings.

fetch_as_element(**kw)[source]

Fetch the blacklist and return as an instance of Element.

Returns:generator returning element instances
Return type:BlacklistEntry

Connections

A connection query returns all currently connected sessions on the given target.

Create a query to obtain all connections for a given engine:

query = ConnectionQuery('sg_vm')

Add a timezone to the query:

query.format.timezone('CST')

Add a filter to only get connections if the source address is 172.18.1.252:

query.add_in_filter(FieldValue(LogField.SRC), [IPValue('172.18.1.252')])

Only connections that match a specific service:

query.add_in_filter(FieldValue(LogField.SERVICE), [ServiceValue('TCP/443', 'UDP/53')])

Execute query and return raw results:

for records in query.fetch_raw():
    ...

Execute query and return as an Connection element:

for records in query.fetch_as_element():
    ...

Retrieving live streaming results:

for records in query.fetch_live():
    ...

See also

smc_monitoring.models.filters for more information on creating filters

class smc_monitoring.monitors.connections.Connection(**data)[source]

Bases: object

Connection represents a state table entry. This is the result of making a ConnectionQuery and using fetch_as_element().

dest_addr

Destination address for this entry

Return type:str
dest_port

Destination port for the entry.

Return type:int
engine

The engine/cluster for this state table entry

Returns:engine or cluster for this entry
Return type:str
protocol

Protocol for this entry

Returns:protocol (UDP/TCP/ICMP, etc)
Return type:str
service

Service for this entry

Returns:service (HTTP/HTTPS, etc)
Return type:str
source_addr

Source address for this entry

Return type:str
source_port

Source port for the entry.

Return type:int
state

State of the connection.

Returns:state, i.e. UDP established, TCP established, etc.
Return type:str
timestamp

Timestamp of this connection. It is recommended to set the timezone on the query to view this timestamp in the systems local time. For example:

query.format.timezone('CST')
Returns:timestamp in string format
Return type:str
class smc_monitoring.monitors.connections.ConnectionQuery(target, **kw)[source]

Bases: smc_monitoring.models.query.Query

Show all current connections on the specified target.

Variables:field_ids (list) – field IDs are the default fields for this entry type and are constants found in smc_monitoring.models.constants.LogField
Parameters:target (str) – name of target engine/cluster
fetch_as_element(**kw)[source]

Fetch the results and return as a Connection element. The original query is not modified.

Returns:generator of elements
Return type:Connection

Logs

LogQuery provides an interface to the SMC Log Viewer to retrieve data in real time or by batch.

There are a variety of settings you can configure on a query such as whether to execute a real time query versus a stored log fetch, time frame for the query, fetch size quantity, returned format style, specify which fields to return and adding filters to make a very specific query.

To make queries, first obtain a query object and optionally (recommended) specify a maximum number of records to fetch (for non-real time fetches). The default log query type is ‘stored’, and if a fetch_size is not provided, one batch of 200 records will be returned:

query = LogQuery(fetch_size=50)

If real time logs are preferred and set fetch_type='current' (default is fetch ‘stored’ logs):

query = LogQuery(fetch_type='current')

You can also use the shortcut fetch_live on the query:

query = LogQuery()
for result in query.fetch_live():
    ...

Note

If selecting fetch_size='current' log queries will be real-time and ignore the fetch_size, time_range, and backwards values if provided on the query.

You can also set a time_range on the query. There are convenience methods on a TimeFormat object to simplify adding a time range. When using time ranges, you should set the timezone on the query to the clients timezone:

query = LogQuery(fetch_size=50)
query.time_range.last_five_minutes()
query.format.timezone('CST')

You can also use custom time ranges to search between a specific period of time. This is done by providing a smc_monitoring.models.calendar.TimeFormat instance to the Query constructor, or by modifying the query time_range attribute. The TimeFormat object takes a ‘naive’ datetime object for start and end times. The start and end times must also be in milliseconds.

Example of finding all records on 9/2/2017 from 06:25:30 to 06:26:30 in the local time zone CST:

dt_start = datetime(2017, 9, 2, 6, 25, 30, 0)
dt_end = datetime(2017, 9, 2, 6, 26, 30, 0)

query = LogQuery()
query.format.timezone('CST')    # <--- Set the timezone on the query!
query.time_range.custom_range(
    datetime_to_ms(dt_start),
    datetime_to_ms(dt_end))

See also

smc_monitoring.models.calendar.TimeFormat for more examples and information on using a TimeFormat in a query.

Adding filters to a query can be achieved by using add_XX_filter convenience methods or by calling update_filter with the filter object.

For example, customizing the fields returned using query.format.field_ids, and filtering for only HIGH alerts with a source address of 192.168.4.84:

query = LogQuery(fetch_size=10)
query.format.timezone('CST')

query.format.field_ids([LogField.TIMESTAMP, LogField.ACTION, LogField.SRC, LogField.DST])

query.add_and_filter(
    [InFilter(FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.HIGH)]),
     InFilter(FieldValue(LogField.SRC), [IPValue('192.168.4.84')])])

See also

smc.monitoring.filters for information on how to use and combine filters for a query.

class smc_monitoring.monitors.logs.LogQuery(fetch_type='stored', fetch_size=None, backwards=True, format=None, time_range=None, **kw)[source]

Bases: smc_monitoring.models.query.Query

Make a Log Query to the SMC to fetch stored log data or monitor logs in real time.

Variables:

field_ids (list) – field IDs are the default fields for this entry type and are constants found in smc_monitoring.models.constants.LogField
Parameters:
  • fetch_type (str) – ‘stored’ or ‘current’
  • fetch_size (int) – max number of logs to fetch
  • backwards (bool) – by default records are returned from newest to oldest (backwards=True). To return in opposite direction, set backwards=False. Default: True
  • format (format type from smc_monitoring.models.formats (default: TextFormat)) – A format object specifying format of return data
  • time_range (TimeFormat) – time filter to add to query
fetch_batch(formatter=<class 'smc_monitoring.models.formatters.TableFormat'>)[source]

Fetch a batch of logs and return using the specified formatter. Formatter is class type defined in smc_monitoring.models.formatters. This fetch type will be a single shot fetch (this method forces fetch_type='stored'). If fetch_size is not already set on the query, the default fetch_size will be 200.

Parameters:formatter – Formatter type for data representation. Any type in smc_monitoring.models.formatters.
Returns:generator returning data in specified format
fetch_live(formatter=<class 'smc_monitoring.models.formatters.TableFormat'>)[source]

View logs in real-time. If previous filters were already set on this query, they will be preserved on the original instance (this method forces fetch_type='current').

Parameters:formatter – Formatter type for data representation. Any type in smc_monitoring.models.formatters.
Returns:generator of formatted results
fetch_raw()[source]

Execute the query and return by batches. Optional keyword arguments are passed to Query.execute(). Whether this is real-time or stored logs is dependent on the value of fetch_type.

Returns:generator of dict results
fetch_size

Return the fetch size for this query. If fetch size is set to 0, the query will be aborted after the first response message. If the fetch_size is None, it is considered undefined which indicates there is no fetch bound set on this query (i.e. fetch all).

..note:: It is recommended to provide a fetch_size to limit the
results when doing a ‘stored’ query.
Returns:configured fetch size for this query
Return type:int

Routes

Query the current routing table entries.

Create a query to obtain all connections for a given engine:

query = RoutingQuery('sg_vm')

Add a timezone to the query:

query.format.timezone('CST')

Add a filter to only routes for destination network 192.168.4.0/24:

query.add_in_filter(FieldValue(LogField.ROUTENETWORK), [IPValue('192.168.4.0')])

Only routes that use a specific gateway:

query.add_in_filter(FieldValue(LogField.ROUTEGATEWAY), [IPValue('172.18.1.200')])

Execute query and return raw results:

for records in query.fetch_batch():
    ...

Execute query and return as an RoutingView element:

for records in query.fetch_as_element():
    ...

See also

smc_monitoring.models.filters for more information on creating filters

class smc_monitoring.monitors.routes.RoutingQuery(target, **kw)[source]

Bases: smc_monitoring.models.query.Query

Show all current dynamic and static routes on the specified target.

Variables:field_ids (list) – field IDs are the default fields for this entry type and are constants found in smc_monitoring.models.constants.LogField
Parameters:target (str) – name of target engine/cluster
fetch_as_element(**kw)[source]

Fetch the results and return as a RoutingView element. The original query is not modified.

Returns:generator of elements
Return type:RoutingView
class smc_monitoring.monitors.routes.RoutingView(**data)[source]

Bases: object

A Routing View represents an entry in the current routing table. This is the result of making a RoutingQuery and using fetch_as_element().

dest_if

Destination interface for this route

Return type:str
dest_vlan

Destination VLAN for this route, if any.

Return type:str
dest_zone

Destination zone for this route, if any.

Return type:str
engine

The engine/cluster for this route

Return type:str
route_gw

The route gateway for this route.

Return type:str
route_metric

Metric for this route.

Returns:route metric
Return type:int
route_network

The route network for this route.

Return type:str
route_type

The type of route.

Returns:Static, Connection, Dynamic, etc.
Return type:str
timestamp

Timestamp of this connection. It is recommended to set the timezone on the query to view this timestamp in the systems local time. For example:

query.format.timezone('CST')

:return timestamp in string format :rtype: str

SSLVPN

SSLVPN currently connected users.

Create a query to obtain all connections for a given engine:

query = SSLVPNQuery('sg_vm')

Add a timezone to the query:

query.format.timezone('CST')

Execute query and return raw results:

for records in query.fetch_batch():
    ...

Execute query and return as an SSLVPNUser element:

for records in query.fetch_as_element():
    ...

See also

smc_monitoring.models.filters for more information on creating filters

class smc_monitoring.monitors.sslvpn.SSLVPNQuery(target, **kw)[source]

Bases: smc_monitoring.models.query.Query

Show all current SSL VPN connections on the specified target.

Variables:field_ids (list) – field IDs are the default fields for this entry type and are constants found in smc_monitoring.models.constants.LogField
Parameters:target (str) – name of target engine/cluster
fetch_as_element(**kw)[source]

Fetch the results and return as an SSLVPNUser element. The original query is not modified.

Returns:generator of elements
Return type:SSLVPNUser
class smc_monitoring.monitors.sslvpn.SSLVPNUser(**data)[source]

Bases: object

Connection represents a state table entry. This is the result of making a SSLVPNQuery and using fetch_as_element().

engine

The engine/cluster for this state table entry

Returns:engine or cluster for this entry
Return type:str
session_expiration

Time the session expires. It is recommended that you add a timezone to the query to present this in human readable format:

query.format.timezone('CST')
Return type:str
session_start

Time the session started. It is recommended that you add a timezone to the query to present this in human readable format:

query.format.timezone('CST')
Return type:str
source_addr

Source IP address for the SSL VPN user

Return type:str
username

Username for this SSL VPN user

Return type:str

Users

Get active users on target cluster/engine.

Create a query to obtain all users for a given engine:

query = UserQuery('sg_vm')

Add a timezone to the query:

query.format.timezone('CST')

Execute query and return raw results:

for records in query.fetch_batch():
    ...

Execute query and return as a User element:

for records in query.fetch_as_element():
    ...

See also

smc_monitoring.models.filters for more information on creating filters

class smc_monitoring.monitors.users.User(**data)[source]

Bases: object

User mapping currently in user cache on specified target. This is the result of making a UserQuery and using fetch_as_element().

domain

SMC Domain that this user record belongs to

Returns:name of SMC domain, ‘Shared’ is default
Return type:str
engine

The engine/cluster for this state table entry

Returns:engine or cluster for this entry
Return type:str
expiration

Expiration time for this user entry. It is recommended to add a timezone to the query to display this field in the client local time.

Returns:expiration time for this user authentication entry
Return type:str
ipaddress

IP address for the entry

Return type:str
timestamp

Timestamp of this connection. It is recommended to set the timezone on the query to view this timestamp in the systems local time. For example:

query.format.timezone('CST')

:return timestamp in string format :rtype: str

username

Username for entry

Returns:username value as fully qualified domain name
Return type:str
class smc_monitoring.monitors.users.UserQuery(target, **kw)[source]

Bases: smc_monitoring.models.query.Query

Show all authenticated users on the specified target.

Variables:field_ids (list) – field IDs are the default fields for this entry type and are constants found in smc_monitoring.models.constants.LogField
Parameters:target (str) – name of target engine/cluster
fetch_as_element(**kw)[source]

Fetch the results and return as a User element. The original query is not modified.

Returns:generator of elements
Return type:User

VPNs

Get all active VPN SA’s.

Create a query to obtain all connections for a given engine:

query = VPNSAQuery('sg_vm')

Add a timezone to the query:

query.format.timezone('CST')

Execute query and return raw results:

for records in query.fetch_batch():
    ...

Execute query and return as a VPNSecurityAssoc element:

for records in query.fetch_as_element():
    ...

Delete a VPN SA:

query = VPNSAQuery('sg_vm')
for sa in query.fetch_as_element():
    sa.delete()

See also

smc_monitoring.models.filters for more information on creating filters

class smc_monitoring.monitors.vpns.VPNSAQuery(target, **kw)[source]

Bases: smc_monitoring.models.query.Query

Show all current VPN SA’s on the specified target.

Variables:field_ids (list) – field IDs are the default fields for this entry type and are constants found in smc_monitoring.models.constants.LogField
Parameters:target (str) – name of target engine/cluster
fetch_as_element(**kw)[source]

Fetch the results and return as a VPNSecurityAssoc element. The original query is not modified.

Returns:generator of elements
Return type:VPNSecurityAssoc
class smc_monitoring.monitors.vpns.VPNSecurityAssoc(**data)[source]

Bases: object

A VPN Security Association represents a currently connected VPN endpoint. This is the result of making a VPNSAQuery and using fetch_as_element().

bytes_received

Number of bytes received.

Return type:int
bytes_sent

Number of bytes sent.

Return type:int
engine

The engine/cluster for this VPN

Return type:str
expiration

Expiration time for this tunnel Security Association

Return type:str
local_endpoint

Local endpoint (IP address) for this VPN tunnel.

Return type:str
local_gateway

Local gateway for this VPN.

Return type:str
local_networks

Local protected networks

Return type:str
negotiation_role

Role for this tunnel entry.

Returns:Negotiation role, i.e. Initiator, Responder, etc.
Return type:str
peer_endpoint

Peer endpoint element and IP Address for this tunnel.

Return type:str
peer_gateway

Peer gateway for this VPN.

Return type:str
peer_networks

Remote protected networks

Return type:str
protocol

WHich protocol is associated with this tunnel entry.

Returns:IP protocol for tunnel, i.e. ESP/UDP
Return type:str
sa_type

SA Type for this VPN tunnel. Each VPN tunnel will typically have at least two entries, one for IPSEC and another for IKE.

Return type:str
timestamp

Timestamp of this connection. It is recommended to set the timezone on the query to view this timestamp in the systems local time. For example:

query.format.timezone('CST')
Return type:str

Alerts

ActiveAlert Query provides the ability to view current alert entries from the alert log viewer. When creating the query, you must specify a target which speifies the SMC domain for which to retrieve the alerts.

A basic alert query using a local timezone example:

query = ActiveAlertQuery('Shared Domain')
query.format.timezone('CST')

You can also use standard filters to specify a more exact match, for example, showing alerts with a severity of CRITICAL:

query.add_in_filter(
    FieldValue(LogField.ALERTSEVERITY), [ConstantValue(Alerts.CRITICAL)])
class smc_monitoring.monitors.alerts.ActiveAlertQuery(target='Shared Domain', timezone=None)[source]

Bases: smc_monitoring.models.query.Query

Active Alert Query is an interface to the alert log viewer in SMC. This query type provides the ability to fetch and filter on active alerts.

You can create a new query specifying a valid timezone abbreviation:

query = ActiveAlertQuery('Shared Domain', timezone='CST')

Or alternatively no timezone:

query = ActiveAlertQuery('DomainFoo')
Parameters:
  • target (str) – domain for which to filter alerts. Default: ‘Shared Domain’
  • timezone (str) – timezone for timestamps, i.e. ‘CST’, etc
fetch_as_element(**kw)[source]

Fetch the results and return as a User element. The original query is not modified.

Returns:generator returning element instances
Return type:Alert
class smc_monitoring.monitors.alerts.Alert(**data)[source]

Bases: object

Alert definition returned from specified domain. This is the result of making a ActiveAlertQuery and using fetch_as_element().

action

Action performed for the alert

Return type:str
destination

Destination IP for the alert

Return type:str
destination_port

Destination port for alert

Return type:int
engine

The engine/cluster for this state table entry

Returns:engine or cluster for this entry
Return type:str
protocol

Protocol for alert

Return type:str
service

Service associated with alert

Return type:str
severity

Severity for this alert

Return type:str
situation

Situation defined for this alert

Return type:str
source

Source IP for the alert

Return type:str
source_port

Source port for alert

Return type:int
timestamp

Timestamp of this connection. It is recommended to set the timezone on the query to view this timestamp in the systems local time. For example:

query.format.timezone('CST')

:return timestamp in string format :rtype: str

vulnerability_refs

Comma seperated string listing any vulnerability references for the alert, if any.

Return type:str