nti.webhooks¶

Webhooks
This Package
- Data Model (Subscription Combinations)
Documentation
TODO
- Dynamic-subscriptions only
- Thoughts on HTTP API

Webhooks ¶

Webhooks are HTTPS requests from one party — the source — to another party, the destination. These requests are one-way: the source sends the request to the destination, and aside from conforming that the request was received, takes no further action (the request’s response from the destination is irrelevant). These requests are sent from the source to let the destination know that something has happened: a new entity (or resource, in the REST sense) has been created, an old one updated or deleted, etc. Such requests typically carry a payload in the body providing information about the action (usually the representation of the affected resource). Destinations are identified via complete URL; destinations may expect to be informed of events affecting one, several, or all possible types of entities handled by the source.

This Package ¶

This package is installed in a source server and manages the registration and sending of webhooks. The registrations may be either static, or they may be dynamic, as in the case of REST Hooks, where individual “subscriptions” may be started and stopped.

This package is intended to integrate with highly event-driven applications using zope.event, that define their resources using zope.interface, manage event delivery, resource adaptation, and dependency injection using zope.component, and (optionally) implement a hierarchy of component registries using zope.site and nti.site. Data persistence is provided through persistent objects, typically with ZODB.

Data Model (Subscription Combinations)¶

One of the motivating examples of this package is integration with Zapier and more generally the notion of REST Hooks.

In this model, a configuration on a server (origin) that sends data to a target URL when events occur is called a subscription. Subscriptions are meant to include:

An event name (or names) the subscription includes;
A parent user or account relationship;
A target URL; and
Active vs inactive state.

Subscription lookup must be performant, so the user and event name information for subscriptions should be fast to find.

Here, event names are defined to “use the noun.verb dot syntax, IE: contact.create or lead.delete).” Using zope.event and zope.component, this translates to the pair of object type or interface, and event type or interface. For example, (IContentContainer, IObjectAddedEvent).

Zapier generates a unique target URL for each event name, so to get created (added), modified, and deleted resources for a single type of object there will be three different target URLs and thus three different subscriptions. In general, there’s an N x M expansion of object types and event types to target URLs or subscriptions.

This package implements this model directly. (You can of course use umbrella interfaces applied to multiple object or event types to send related events to a single subscription.) Aggregating data views of “all webhook deliveries for a type of object” or “all webhook deliveries for a type of event” for presentation purposes could be written, but isn’t particularly natural given how its set up now.

An important outcome of this model is that there’s no need for any given HTTP request to explicitly include something that identifies the type of event; the default dialect (see below) assume that the URL includes everything the receiver needs for that and doesn’t do anything like add an X-NTI-EventType header or add something to the JSON body. It can be a URL parameter or a whole different URL, doesn’t matter.

Note

See the Glossary for common terminology.

Documentation ¶

Scope¶

Out Of Scope¶

Certain concerns are out of scope for this package (but other packages built upon this package my provide them). These include, but are not limited to:

Providing a user interface for managing subscriptions.
Providing an HTTPS API for managing subscriptions. This package provides the underlying data storage, but accepting parameters, etc, and marshaling them into the correct Python calls, is not a concern here.
Providing a user interface or HTTPS API for viewing webhook audit logs.
Enabling webhooks to fire only for specific objects. This package deals with scopes (sites) and kinds of objects, not individual instances.

In Scope/Features¶

Certain concerns are very much in scope for this package, and this package should provide a complete, easy to use solution that addresses these concerns. Where necessary, if a concern cannot be addressed directly by this package, extension points (interfaces and zope.component utilities) may be defined. These include, but are not limited to:

Resource Representation

The on-the-wire form of the resources is built using nti.externalization.

To allow customization of the external forms, a named externalizer is used; nti.externalization will fall back to the default externalizer if no externalizer of the given name is available. The default externalizer is named “webhook-delivery”, but dialects may use something different.
Alternate Webhook Dialects

Webhooks are a general protocol and mostly interoperable. But to support cases where particular destinations have specific requirements, “dialects” are used. There is a default dialect and then there may be specializations of it. Each webhook subscription may have associated with it the name of a dialect to use. These dialects are found in the component registry. For example, a dialect may choose to use a different externalizer name such as “zapier-webhook-delivery”.
Transactional

Webhooks should not be delivered if the ultimate creation or persistence of a resource failed. To this end, webhook delivery in this package is integrated with the transaction package.

Resources are externalized during a late phase of the transaction commit process; the details about the delivery are recorded and persisted, and only after the transaction is successfully committed does the HTTP request get made.
Concurrency

Webhook delivery and record keeping should be lightweight, and all actual network IO should proceed in a non-blocking fashion. This means that this package will spawn threads (or greenlets, using gevent.
Error Handling/Failure Retry

A limited amount of retry logic is provided by this package, but that does not extend to process boundaries. If the process hosting this package is killed while a delivery is pending, no automatic provision is made to resume delivery attempts in any other process.

The API is present to allow that to be implemented, though.
Auditing/Delivery History

For each subscription, delivery attempts, status, and responses are stored in a ring-buffer like structure. This can be inspected to see if deliveries succeeded, failed, or never completed.
Access Control on Deliveries

Each subscription is associated with an IPrincipal that owns it. A request is only delivered to a subscription if the IPrincipal that owns the subscription can access the entity, as determined by zope.security.
Access Control on Subscriptions

While not enforced by this package, the above owner relationship will be used to provide role managers that grant read and read/write access to remove subscriptions only to the owner of the subscription.

TODO: Make sure client packages can extend that to provide for admin access. So long as we don’t DENY it should be fine.
Hierarchy of Subscriptions

Subscriptions are made within a particular Zope site (the closest enclosing site to a resource when a resource is subscribed to, or the currently active site otherwise). These sites may have parents.

TODO: Work out the details of that.

When an event is received that might result in webhook delivery, active subscriptions are checked for in the currently active site, as well as in the sites up the hierarchy of the resource itself. All applicable subscribers will get a delivery.

For example, if the president of the company (an administrator) subscribes to “new user created” events at the global (root, base or “/”) level, and a department head subscribes to “new user created” for their department (“/NOAA”), while a local office manager subscribes to events for their office (“/NOAA/NWS/OUN”), then creating a new user in the OKC office may send three deliveries, one to the manager, one to the secretary, and one to the president.

Note

If there are identical subscribed URLs with differing permission requirements, then if access is granted for any subscription, the payload will be delivered.

Note

While looking up both the resource and active site tree might seem complex, following both hierarchies is necessary in the event of operations that span multiple child sites. This is probably most common with bulk operations, but a simple example would be the president logging in to the root site, searching for and deleting all employees named “Bill.” If one was in the OKC office and one was in the OUN office, the managers of both locations should get delivery.
Converting From Object Events to Webhook Events

TODO: Write me.

This package needs to have a clear way to have client packages specify what events should produce webhook deliveries. The exact mechanism is TBD. Possibly clients are expected to use <classImplements> ZCML directives to apply marker interfaces? Or they might register a subscriber provided by this package for their own existing interfaces?

We want this process, and the process of finding all active subscriptions, to be fast. I’m imagining something like view lookup, keeping active subscriptions in the various component registries? That doesn’t work non-persistently.

Glossary¶

active

Of a subscription: An existing subscription is active if it is ready to accept webhook deliveries. Contrast with inactive. A subscription in this state may transaction to inactive at any time.

applicable

Of a subscription: Does the subscription apply to some piece of data, including permission checks? Only active subscriptions should be applicable.

dialect

A customization point for a subscription. A dialect is used to create and populate the HTTP request.

externalized

Of an object: Written out in a format suitable for use in an HTTP REST API, such as JSON, using nti.externalization.

inactive

Of a subscription: An existing subscription is inactive if webhook deliviries will no longer be attempted to it. It may transition back to active at any time.

subscription

A target to which webhooks will be delivered.

Subscriptions may be either active or inactive. Only active subscriptions will result in delivery attempts.

In addition to capturing the target URL and HTTP method to use, a subscription knows the dialect to use, along with the security restrictions to apply. It also has a delivery history.

See also

IWebhookSubscription

target

Of a subscription: The URL to which the HTTP request is sent.

trigger

An zope.interface.interfaces.IObjectEvent, when notified through zope.event.notify() may trigger a matching subscription to attempt a delivery.

Configuration¶

nti.webhooks uses zope.configuration.xmlconfig and ZCML files for basic configuration.

Loading the default configure.zcml for this package establishes some defaults, such as the default global webhook delivery manager.

>>> from zope.configuration import xmlconfig
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" />
... </configure>
... """)

Important

By itself, that is not enough for this package to be functional.

The subscriber nti.webhooks.subscribers.dispatch_webhook_event() must be registered as well, for some combination of data object and (descendent of) zope.interface.interfaces.IObjectEvent.

Note

Descending from IObjectEvent is not actually required, so long as the event provides the object attribute, and so long as double-dispatch from a single event to the double (object, event) subscriber interface happens. This is automatic for IObjectEvent.

Because IObjectEvent and its descendents are extremely common events, that subscriber is not registered by default. Doing so could add unacceptable overhead to common application actions. It is suggested that your application integration should register the subscriber for a subset of “interesting” events and data types.

Your application integration is free to register the subscriber for exactly the events that are desired. Or, to assist with common cases, this package provides two additional ZCML files.

Recommended: `subscribers.zcml`¶

This file registers the subscriber for commonly useful object lifecycle events:

Note

The IObjectCreatedEvent is specifically not registered. While this is the first event typically sent during an object’s lifecycle, when it is fired, the object is not required to have a location (__name__ and __parent__) yet. It also typically does not have proper security constraints yet (which are usually location dependent). This means that URLs cannot be generated for it, nor can security be enforced.

Rather than register those for *, meaning any object, those are registered for nti.webhooks.interfaces.IPossibleWebhookPayload. This marker interface is meant to be mixed in by the application to classes that are subject to events and for which webhook delivery may be desired.

Note

This is separate from nti.webhooks.interfaces.IWebhookPayload, which is used as an adapter from an object delivered with an event to the object that should actually be externalized for delivery of the event.

For example:

>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers.zcml" />
...   <class class="nti.webhooks.testing.InterestingClass">
...      <implements interface="nti.webhooks.interfaces.IPossibleWebhookPayload" />
...   </class>
... </configure>
... """)

Development and Testing: `subscribers_promiscuous.zcml`¶

This file registers the dispatcher for all object events for all objects: (*, IObjectEvent*).

This may have performance consequences, so its use in production systems is discouraged (unless the system is small). However, it is extremely useful during development and (unit) testing and while deciding which objects and events make useful webhooks.

Many of the tests and examples in the documentation for this package use this file.

For example:

>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
... </configure>
... """)

Configured Global, Transient Webhook Subscriptions¶

The simplest type of webhook subscription is one that is configured statically, typically at application startup time, and stores no persistent history (with the facilities provided by this package; applications may store their own history, perhaps by listening for delivery events).

This is useful for a number of scenarios, including:

Development;
Integration testing;
Fire-and-forget delivery of frequent events;
Simple applications.

This package provides ZCML directives to facilitate this. The directives can either be used globally, creating subscriptions that are valid across the entire application.

interface nti.webhooks.zcml.IStaticSubscriptionDirective[source]¶

Define a global, static, transient subscription.

Static subscriptions are not persistent and live only in the memory of individual processes. Thus, failed deliveries cannot be re-attempted after process shutdown. And of course the delivery history is also transient and local to a process.

for_¶

The type of object to attempt delivery for.

When object events of type when are fired for instances providing this interface, webhook delivery to target might be attempted.

The default is objects that implement IWebhookPayload.

This is interpreted as for zope.component.registerAdapter() and may name an interface or a type.

Implementation:	`zope.configuration.fields.GlobalObject`
Read Only:	False
Required:	False
Default Value:	<InterfaceClass nti.webhooks.interfaces.IWebhookPayload>

when¶

The type of event that should result in attempted deliveries.

A type of IObjectEvent, usually one defined in zope.lifecycleevent.interfaces such as IObjectCreatedEvent. The object field of this event must provide the for_ interface; it’s the data from the object field of this event that will be sent to the webhook.

If not specified, all object events involving the for_ interface will be sent.

This must be an interface.

Implementation:	`nti.webhooks._schema.ObjectEventInterface`
Read Only:	False
Required:	False
Default Value:	<InterfaceClass zope.interface.interfaces.IObjectEvent>

Value Type

Implementation:	`nti.webhooks._schema.ObjectEventField`
Read Only:	False
Required:	True
Default Value:	None

to¶

The complete destination URL to which the data should be sent

This is an arbitrary HTTPS URL. Only HTTPS is supported for delivery of webhooks.

Implementation:	`nti.webhooks._schema.HTTPSURL`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

dialect¶

The ID of the IWebhookDialect to use

Dialects are named utilities. They control the authentication, headers, and HTTP method.

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

owner¶

The ID of the IPrincipal that owns this subscription.

This will be validated at runtime when an event arrives. If the current zope.security.interfaces.IAuthentication utility cannot find a principal with the given ID, the delivery will be failed.

Leave unset to disable security checks.

This cannot be changed after creation.

Implementation:	`nti.webhooks._schema.PermissivePrincipalId`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

permission¶

The permission to check

If given, and an owner is also specified, then only data that has this permission for the owner will result in an attempted delivery. If not given, but an owner is given, this will default to the standard view permission ID, zope.View.

Implementation:	`zope.security.zcml.Permission`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

Let’s look at an example of how to use this directive from ZCML. We need to define the XML namespace it’s in, and we need to include the configuration file that defines it. We also need to have the event dispatching provided by zope.component properly set up, as well as some other things described in Configuration. Including this package’s configuration handles all of that.

>>> from zope.configuration import xmlconfig
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
... </configure>
... """, execute=False)

Once that’s done, we can use the webhooks:staticSubscription XML tag to define a subscription to start receiving our webhook deliveries.

ZCML Directive Arguments¶

There is only one required argument: the destination URL.

The destination must be HTTPS.

>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <webhooks:staticSubscription to="http://example.com" />
... </configure>
... """, conf_context)
Traceback (most recent call last):
...
zope.configuration.exceptions.ConfigurationError: Invalid value for 'to'
    File "<string>", line 6.2-6.57
    zope.schema.interfaces.InvalidURI: http://example.com

If we specify a permission to check, it must exist.

>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <webhooks:staticSubscription
...             to="https://example.com"
...             permission="no.such.permission" />
... </configure>
... """, conf_context)
Traceback (most recent call last):
...
zope.configuration.config.ConfigurationExecutionError: File "<string>", line 6.2-8.46
  Could not read source.
    ValueError: ('Undefined permission ID', 'no.such.permission')

Specifying Which Objects¶

The above (unsuccessful) registration would have tried to send all IObjectEvent events for all objects that implement IWebhookPayload to https://example.com using the default dialect. That’s unlikely to be what you want, outside of tests. Instead, you’ll want to limit the event to particular kinds of objects, and particular events in their lifecycle. The for and when attributes let you do that. Here, we’ll give a comple example saying that whenever a new IContainer is created, we’d like to deliver a webhook.

>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.component" />
...   <include package="zope.container" />
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <webhooks:staticSubscription
...             to="https://this_domain_does_not_exist"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectCreatedEvent" />
... </configure>
... """)

Note

Sometimes, the for and when attributes may not quite get you what you want. You can use an adapter to nti.webhooks.interfaces.IWebhookPayload to derive the desired data. For more, see Customizing HTTP Requests.

Active Subscriptions¶

Now that we have that in place, let’s verify that it exists as part of the global default IWebhookSubscriptionManager.

>>> from nti.webhooks.interfaces import IWebhookSubscriptionManager
>>> from zope import component
>>> sub_manager = component.getUtility(IWebhookSubscriptionManager)
>>> from zope.interface import verify
>>> verify.verifyObject(IWebhookSubscriptionManager, sub_manager)
True
>>> len(list(sub_manager))
1
>>> name, subscription = list(sub_manager.items())[0]
>>> print(name)
Subscription
>>> subscription
<...Subscription ... to='https://this_domain_does_not_exist' for=IContentContainer when=IObjectCreatedEvent>

And we’ll verify that it is active, by looking for it using the event we just declared:

>>> from nti.webhooks.subscribers import find_active_subscriptions_for
>>> from zope.container.folder import Folder
>>> from zope.lifecycleevent import ObjectCreatedEvent
>>> event = ObjectCreatedEvent(Folder())
>>> active_subscriptions = list(find_active_subscriptions_for(event.object, event))
>>> len(active_subscriptions)
1
>>> active_subscriptions[0]
<...Subscription ... to='https://this_domain_does_not_exist' for=IContentContainer when=IObjectCreatedEvent>
>>> active_subscriptions[0] is subscription
True
>>> subscription.active
True

Next, we need to know if the subscription is applicable to the data. Since we didn’t specify a permission or a principal to check, the subscription is applicable:

See also

Delivery Security Checks for information on security checks.

>>> subscriptions = find_active_subscriptions_for(event.object, event)
>>> [subscription.isApplicable(event.object) for subscription in subscriptions]
[True]

Delivery Attempts¶

All attempts at delivering a webhook are recorded. Delivery always occurs as a result of committing a transaction, and the resulting attempt object is stored in the corresponding subscription object.

Here, we will briefly look at what happens when we attempt to deliver this webhook. Recall that it uses a domain that does not exist.

See also

Delivery History for more on delivery attempts.

See also

Customizing HTTP Requests for information on customizing what is sent in the delivery attempt.

Unsuccessful Delivery Attempts¶

Because delivery is transactional, to begin we must be in a transaction:

>>> import transaction
>>> tx = transaction.begin()

Fire the event:

>>> from zope import lifecycleevent
>>> lifecycleevent.created(Folder())

We can see that we have attached a data manager to the transaction:

>>> tx._resources
[<nti.webhooks.datamanager.WebhookDataManager...>]

Don’t Fail The Transaction¶

However, recall that we specified an invalid domain name, so there is nowhere to attempt to deliver the webhook too. For static webhooks, this is generally a deployment configuration problem and should be attended to by correcting the ZCML. For dynamic subscriptions, the error would be corrected by updating the subscription. This doesn’t fail the commit:

>>> transaction.commit()

But it does record a failed attempt in the subscription:

>>> subscription = sub_manager['Subscription']
>>> len(subscription)
1
>>> attempt = list(subscription.values())[0]
>>> attempt.status
'failed'
>>> print(attempt.message)
Verification of the destination URL failed. Please check the domain.
>>> len(attempt.internal_info.exception_history)
1
>>> print(attempt.internal_info.exception_history[0])
Traceback (most recent call last):
...

Inactive Subscriptions¶

Subscriptions can be deactivated (made inactive) by asking the manager to do this. The subscription manager is always the subscription’s parent, and deactivating the subscription more than once does nothing.

>>> subscription.__parent__ is sub_manager
True
>>> sub_manager.deactivateSubscription(subscription)
True
>>> sub_manager.deactivateSubscription(subscription)
False
>>> subscription.active
False

Note that we cannot change this attribute directly, it must be done through the manager.

>>> subscription.active = True
Traceback (most recent call last):
...
ValueError:...field is readonly

Inactive subscriptions will not be used for future deliveries, but their existing history is preserved.

>>> len(subscription)
1
>>> find_active_subscriptions_for(event.object, event)
[]
>>> tx = transaction.begin()
>>> lifecycleevent.created(Folder())
>>> tx._resources
[]
>>> transaction.commit()
>>> len(subscription)
1

Of course, inactive subscriptions can be activated again.

>>> sub_manager.activateSubscription(subscription)
True
>>> subscription.active
True
>>> tx = transaction.begin()
>>> lifecycleevent.created(Folder())
>>> transaction.commit()
>>> len(subscription)
2

Removing a subscription from its subscription manager automatically deactivates it.

>>> del sub_manager[subscription.__name__]
>>> subscription.__parent__ is None
True
>>> subscription.active
False

Configured Local, Persistent Webhook Subscriptions¶

A step between global, static, transient subscriptions and local, runtime-installed history-preserving subscriptions are the subscriptions described in this document: they are statically configured using ZCML, but instead of being global, they are located in the database (in a site manager) and store history.

The ZCML directive is very similar to IStaticSubscriptionDirective.

interface nti.webhooks.zcml.IStaticPersistentSubscriptionDirective[source]

Extends: nti.webhooks.zcml.IStaticSubscriptionDirective

Define a local, static, persistent subscription.

Local persistent subscriptions live in the ZODB database, beneath some zope.site.interfaces.ILocalSiteManager.

They are identified by a traversable path beginning from the root of the database; note that this may not be the exact same as a path exposed in the application because this path will need to include the name of the root application object, while application paths typically do not.

This package uses zope.generations to keep track of registered subscriptions and synchronize the database with what is in executed ZCML. Thus it is very important not to remove ZCML directives, or only execute part of the ZCML configuration unless you intend for the subscriptions not found in ZCML to be removed.

All the options are the same as for IStaticSubscriptionDirective, with the addition of the required site_path.

site_path¶

The path to traverse to the site

A persistent subscription manager will be installed in this site.

Implementation:	`nti.webhooks.zcml.Path`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

In order to use this directive, there must be at least one site manager configured in the main ZODB database. This can be done in a variety of ways, but one of the easiest is to use zope.app.appsetup. We do this by just including its configuration (along with a few standard packages).

>>> from zope.configuration import xmlconfig
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <include package="zope.app.appsetup" />
... </configure>
... """)

Wait, didn’t we forget someting? Where’s the database? Well, we haven’t established one yet. Let’s do that.

>>> from nti.site.testing import print_tree
>>> from nti.webhooks.testing import DoctestTransaction
>>> tx = DoctestTransaction()
>>> db = tx.db
>>> conn = tx.begin()
>>> root = conn.root()
>>> print_tree(root, depth=0, details=('len',))
<Connection Root Dictionary> len=0
>>> tx.finish()

Of course, creating the database by itself does nothing. Like most things, zope.app.appsetup is based on events. Including its configuration just established the event handlers. In this case, the event that needs to be sent is zope.processlifetime.DatabaseOpened.

>>> from zope.processlifetime import DatabaseOpened
>>> from zope.event import notify
>>> notify(DatabaseOpened(db))
>>> def show_trees():
...   def extra_details(obj):
...       from nti.webhooks.interfaces import IWebhookSubscription
...       if IWebhookSubscription.providedBy(obj):
...          return ['to=%s' % (obj.to,), 'active=%s' % (obj.active,)]
...       return ()
...   with tx as conn:
...     root = conn.root()
...     print_tree(root,
...                depth=0,
...                show_unknown=type,
...                details=('len', 'siteManager'),
...                extra_details=extra_details,
...                basic_indent='  ',
...                known_types=(int, tuple,))
>>> show_trees()
<Connection Root Dictionary> len=1
   <ISite,IRootFolder>: Application len=0
     <Site Manager> name=++etc++site len=1
       default len=3
         CookieClientIdManager => <class 'zope.session.http.CookieClientIdManager'>
         PersistentSessionDataContainer len=0
         RootErrorReportingUtility => <class 'zope.error.error.RootErrorReportingUtility'>

The event handler here first made sure there was a site (called “Application”) with a few standard utilities, and then notified zope.processlifetime.DatabaseOpenedWithRoot. That event is used by zope.generations to perform further installation activities, as wall as upgrades and migrations.

And finally, then, that’s here this package comes in. It connects with zope.generations to manage adding, and removing, these persistent local subscriptions.

Adding A Subscription¶

Let’s use the ZCML to add a subscription. The unique required parameter here is site_path, which must be the traversable path to a ISite into which the persistent subscription manager will be installed.

Important

To use zope.generations, and consequently this package’s integration with it, you must either specifically include the subscriber.zcml from that package (which evolves to the minimum required generation), or manually register the alternate handler that evolves to the maximum available generation.

If you manually register the alternate subscriber that simply checks whether the generation is sufficient, you will not be able to make future changes to your persistent webhook subscriptions.

>>> zcml_string = """
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" file="meta.zcml" />
...   <include package="zope.generations" file="subscriber.zcml" />
...   <webhooks:persistentSubscription
...             site_path="/Application"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectModifiedEvent"
...             to="https://example.com" />
... </configure>
... """
>>> _ = xmlconfig.string(zcml_string)

Once again, this hasn’t done anything to the database yet, it’s merely collected the necessary information.

>>> show_trees()
<Connection Root Dictionary> len=1
   <ISite,IRootFolder>: Application len=0
     <Site Manager> name=++etc++site len=1
       default len=3
         CookieClientIdManager => <class 'zope.session.http.CookieClientIdManager'>
         PersistentSessionDataContainer len=0
         RootErrorReportingUtility => <class 'zope.error.error.RootErrorReportingUtility'>

To take action we need to notify the event. When we do, we see that a subscription manager and subscription have been created in the defined location. Also, some bookkeeping information has been added to the root of the database.

>>> notify(DatabaseOpened(db))
>>> show_trees()
<Connection Root Dictionary> len=3
   <ISite,IRootFolder>: Application len=0
     <Site Manager> name=++etc++site len=1
       default len=4
         CookieClientIdManager => <class 'zope.session.http.CookieClientIdManager'>
         PersistentSessionDataContainer len=0
         RootErrorReportingUtility => <class 'zope.error.error.RootErrorReportingUtility'>
         ZCMLWebhookSubscriptionManager len=1
           PersistentSubscription len=0 to=https://example.com active=True
   nti.webhooks.generations.PersistentWebhookSchemaManager => <class 'nti.webhooks.generations.State'>
   zope.generations len=1
     zzzz-nti.webhooks => 1

The bookkeeping information is used to make sure that subscriptions in the database stay in sync with what’s in the ZCML. If we execute the same ZCML again and re-notify the database opening, nothing in the database changes.

>>> _ = xmlconfig.string(zcml_string)
>>> notify(DatabaseOpened(db))
>>> show_trees()
<Connection Root Dictionary> len=3
   <ISite,IRootFolder>: Application len=0
     <Site Manager> name=++etc++site len=1
       default len=4
         CookieClientIdManager => <class 'zope.session.http.CookieClientIdManager'>
         PersistentSessionDataContainer len=0
         RootErrorReportingUtility => <class 'zope.error.error.RootErrorReportingUtility'>
         ZCMLWebhookSubscriptionManager len=1
           PersistentSubscription len=0 to=https://example.com active=True
   nti.webhooks.generations.PersistentWebhookSchemaManager => <class 'nti.webhooks.generations.State'>
   zope.generations len=1
     zzzz-nti.webhooks => 1

Delivering To The Subscription¶

Delivering to the subscription can happen in two ways. We’ll use the same helper function in both examples.

>>> from nti.testing.time import time_monotonically_increases
>>> from nti.webhooks.testing import wait_for_deliveries
>>> from zope.container.folder import Folder
>>> from zope import lifecycleevent
>>> @time_monotonically_increases
... def deliver_one(content=None):
...       content = Folder() if content is None else content
...       lifecycleevent.modified(content)
>>> from nti.webhooks.testing import mock_delivery_to
>>> mock_delivery_to('https://example.com', method='POST', status=200)

The Site Is The Active Site

First, if that site is the current active site, a matching resource and event will trigger delivery.

>>> from zope.traversing import api as ztapi
>>> from zope.component.hooks import site as active_site
>>> with tx as conn:
...     site = conn.root.Application
...     with active_site(site):
...         deliver_one()
>>> wait_for_deliveries()
>>> show_trees()
<Connection Root Dictionary> len=3
   <ISite,IRootFolder>: Application len=0
     <Site Manager> name=++etc++site len=1
       default len=4
         CookieClientIdManager => <class 'zope.session.http.CookieClientIdManager'>
         PersistentSessionDataContainer len=0
         RootErrorReportingUtility => <class 'zope.error.error.RootErrorReportingUtility'>
         ZCMLWebhookSubscriptionManager len=1
           PersistentSubscription len=1 to=https://example.com active=True
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
   nti.webhooks.generations.PersistentWebhookSchemaManager => <class 'nti.webhooks.generations.State'>
   zope.generations len=1
     zzzz-nti.webhooks => 1

Note how the PersistentSubscription has gained a delivery attempt.

In The Context Of The Site

Second, if something were to happen to an object within the context (beneath) that site, then, no matter what the active site is, delivery will be attempted.

>>> from zope import component
>>> with tx as conn:
...    site = conn.root.Application
...    assert component.getSiteManager() is component.getGlobalSiteManager()
...    site['Folder'] = Folder()
...    deliver_one(site['Folder'])
>>> wait_for_deliveries()
>>> show_trees()
<Connection Root Dictionary> len=3
   <ISite,IRootFolder>: Application len=1
     Folder len=0
     <Site Manager> name=++etc++site len=1
       default len=4
         CookieClientIdManager => <class 'zope.session.http.CookieClientIdManager'>
         PersistentSessionDataContainer len=0
         RootErrorReportingUtility => <class 'zope.error.error.RootErrorReportingUtility'>
         ZCMLWebhookSubscriptionManager len=1
           PersistentSubscription len=3 to=https://example.com active=True
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
   nti.webhooks.generations.PersistentWebhookSchemaManager => <class 'nti.webhooks.generations.State'>
   zope.generations len=1
     zzzz-nti.webhooks => 1

The number of delivery attempts has again grown.

Important

But why did it grow by two new delivery attempts? We only tried to deliver one event.

The answer is that adding the folder to the site first sent a zope.container.contained.ContainerModifiedEvent for the site folder itself, and then we sent a modified event for the folder we just created. The site and its ContainerModifiedEvent matched our subscription filter.

This is a reminder to be careful about the subscription filters or the Configuration you choose.

Neither Of The Above

Finally, we’ll prove that if the site isn’t the current site, and the object being modified isn’t in the context of that site, no delivery is attempted.

>>> with tx as conn:
...    assert component.getSiteManager() is component.getGlobalSiteManager()
...    deliver_one()
>>> wait_for_deliveries()
>>> show_trees()
<Connection Root Dictionary> len=3
   <ISite,IRootFolder>: Application len=1
     Folder len=0
     <Site Manager> name=++etc++site len=1
       default len=4
         CookieClientIdManager => <class 'zope.session.http.CookieClientIdManager'>
         PersistentSessionDataContainer len=0
         RootErrorReportingUtility => <class 'zope.error.error.RootErrorReportingUtility'>
         ZCMLWebhookSubscriptionManager len=1
           PersistentSubscription len=3 to=https://example.com active=True
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
   nti.webhooks.generations.PersistentWebhookSchemaManager => <class 'nti.webhooks.generations.State'>
   zope.generations len=1
     zzzz-nti.webhooks => 1

As we can see, nothing changed.

Mutating Subscriptions¶

Over time, the ZCHL configuration is likely to change. Subscriptions will be added, removed, or (in rare cases) updated.

Tip

Note that subscriptions are identified by their parameters in the ZCML. Changing any of those parameters counts as a new subscription and a deactivation of the old subscription.

When that happens, the schema manager will make the appropriate adjustments. For additions, a new subscription will be created; existing subscriptions will be unchanged.

>>> zcml_string = """
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" file="meta.zcml" />
...   <include package="zope.generations" file="subscriber.zcml" />
...   <webhooks:persistentSubscription
...             site_path="/Application"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectModifiedEvent"
...             to="https://example.com" />
...   <webhooks:persistentSubscription
...             site_path="/Application"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectModifiedEvent"
...             to="https://example.com/another/path" />
... </configure>
... """
>>> _ = xmlconfig.string(zcml_string)
>>> notify(DatabaseOpened(db))
>>> show_trees()
<Connection Root Dictionary> len=3
   <ISite,IRootFolder>: Application len=1
     Folder len=0
     <Site Manager> name=++etc++site len=1
       default len=4
         CookieClientIdManager => <class 'zope.session.http.CookieClientIdManager'>
         PersistentSessionDataContainer len=0
         RootErrorReportingUtility => <class 'zope.error.error.RootErrorReportingUtility'>
         ZCMLWebhookSubscriptionManager len=2
           PersistentSubscription len=3 to=https://example.com active=True
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
           PersistentSubscription-2 len=0 to=https://example.com/another/path active=True
   nti.webhooks.generations.PersistentWebhookSchemaManager => <class 'nti.webhooks.generations.State'>
   zope.generations len=1
     zzzz-nti.webhooks => 2

Notice the addition of a new subscription, and the increment of the generation.

Now we’ll try something a bit more complex. We’ll add a new subscription above the existing subscriptions, “mutate” one of the existing subscriptions, and completely remove one by commenting it out.

>>> zcml_string = """
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" file="meta.zcml" />
...   <include package="zope.generations" file="subscriber.zcml" />
...   <webhooks:persistentSubscription
...             site_path="/Application"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectModifiedEvent"
...             to="https://example.com/ThisIsNew" />
...   <!-- Comment out this one
...   <webhooks:persistentSubscription
...             site_path="/Application"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectModifiedEvent"
...             to="https://example.com" />
...   -->
...   <webhooks:persistentSubscription
...             site_path="/Application"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectModifiedEvent"
...             to="https://example.com/another/path" />
... </configure>
... """
>>> _ = xmlconfig.string(zcml_string)
>>> notify(DatabaseOpened(db))
>>> show_trees()
<Connection Root Dictionary> len=3
   <ISite,IRootFolder>: Application len=1
     Folder len=0
     <Site Manager> name=++etc++site len=1
       default len=4
         CookieClientIdManager => <class 'zope.session.http.CookieClientIdManager'>
         PersistentSessionDataContainer len=0
         RootErrorReportingUtility => <class 'zope.error.error.RootErrorReportingUtility'>
         ZCMLWebhookSubscriptionManager len=3
           PersistentSubscription len=3 to=https://example.com active=False
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
           PersistentSubscription-2 len=0 to=https://example.com/another/path active=True
           PersistentSubscription-3 len=0 to=https://example.com/ThisIsNew active=True
   nti.webhooks.generations.PersistentWebhookSchemaManager => <class 'nti.webhooks.generations.State'>
   zope.generations len=1
     zzzz-nti.webhooks => 3

We can see the addition of a new subscription. The one we deleted is still present in the database, but in fact it was deactivated. What happens if we uncomment it?

>>> zcml_string = """
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" file="meta.zcml" />
...   <include package="zope.generations" file="subscriber.zcml" />
...   <webhooks:persistentSubscription
...             site_path="/Application"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectModifiedEvent"
...             to="https://example.com/ThisIsNew" />
...   <webhooks:persistentSubscription
...             site_path="/Application"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectModifiedEvent"
...             to="https://example.com" />
...   <webhooks:persistentSubscription
...             site_path="/Application"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectModifiedEvent"
...             to="https://example.com/another/path" />
... </configure>
... """
>>> _ = xmlconfig.string(zcml_string)
>>> notify(DatabaseOpened(db))
>>> show_trees()
<Connection Root Dictionary> len=3
   <ISite,IRootFolder>: Application len=1
     Folder len=0
     <Site Manager> name=++etc++site len=1
       default len=4
         CookieClientIdManager => <class 'zope.session.http.CookieClientIdManager'>
         PersistentSessionDataContainer len=0
         RootErrorReportingUtility => <class 'zope.error.error.RootErrorReportingUtility'>
         ZCMLWebhookSubscriptionManager len=4
           PersistentSubscription len=3 to=https://example.com active=False
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
             ... => <class 'nti.webhooks.attempts.PersistentWebhookDeliveryAttempt'>
           PersistentSubscription-2 len=0 to=https://example.com/another/path active=True
           PersistentSubscription-3 len=0 to=https://example.com/ThisIsNew active=True
           PersistentSubscription-4 len=0 to=https://example.com active=True
   nti.webhooks.generations.PersistentWebhookSchemaManager => <class 'nti.webhooks.generations.State'>
   zope.generations len=1
     zzzz-nti.webhooks => 4

A new subscription is added. No attempt is made to re-activate a previously existing deactivated subscription.

Delivery Security Checks¶

In Configured Global, Transient Webhook Subscriptions, we discussed how to use ZCML to register for webhook deliveries when certain events on certain resources happen. Conspicuously absent from that discussion was any mention of two of the more interesting and important attributes of the ZCML directive: owner and permission. This document will fill in those details.

Owners¶

Owners, for both static subscriptions and other types of subscriptions, are strings naming a IPrincipal as defined in zope.security.interfaces. At runtime, the string is turned into an IPrincipal object using zope.authentication.interfaces.IAuthentication.getPrincipal() from the IAuthentication utility closest to the object of the event (i.e., the for part of the directive).

Note

IAuthentication utilities are usually arranged into a hierarchy related to the object hierarchy, and principals may exist in some parts of the tree and not in others. (While steps such as prefixing the principal ID are usually taken to avoid having the same principal ID be valid in two different parts of the site hierarchy but mean two completely different principals, that is a valid possibility.)

If no principal can be found, the unauthenticated (anonymous) principal will be used instead; depending on the security policy and permission structure in use, the subscription may or may not be applicable.

Permissions¶

Permissions are defined by IPermission, but the important part is how they are checked.

The ISecurityPolicy is used to produce an IInteraction object. The IInteraction is then asked to confirm the permission using its checkPermission(permission, object) method. Normally, the security policy is a global object, and there can only be one active interaction at a time, but temporary, sub-interactions are possible. This module follows that model.

Important

When creating the temporary interactions to check permissions, this module will use the global security policy to create an interaction containing two participations: one for the principal found relating to the object, and one for the principal that was previously participating, if there was one. In this way, the permission must be allowed to both the initiator of the action (the logged in user) as well as the owner of the subscription.

Example¶

Lets put these pieces together and check how security applies.

We’ll begin by defining the same subscription we used previously, but we’ll establish a security policy, and require a permission and owner for the subscription. We’ll use zope.principalregistry to provide a global IAuthentication utility; this also provides a global IUnauthenticatedPrincipal that is used if the owner cannot be found at runtime:

>>> from zope.configuration import xmlconfig
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.component" />
...   <include package="zope.container" />
...   <include package="zope.principalregistry" />
...   <include package="zope.securitypolicy" />
...   <include package="zope.securitypolicy" file="securitypolicy.zcml" />
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <webhooks:staticSubscription
...             to="https://this_domain_does_not_exist"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectCreatedEvent"
...             permission="zope.View"
...             owner="some.one" />
... </configure>
... """)

Next, we can find the active subscription, just as before:

>>> from nti.webhooks.subscribers import find_active_subscriptions_for
>>> from zope.container.folder import Folder
>>> from zope.lifecycleevent import ObjectCreatedEvent
>>> event = ObjectCreatedEvent(Folder())
>>> len(find_active_subscriptions_for(event.object, event))
1
>>> find_active_subscriptions_for(event.object, event)
[<...Subscription ... to='https://this_domain_does_not_exist' for=IContentContainer when=IObjectCreatedEvent>]

Subscription Is Not Applicable By Default¶

Next, we need to know if the subscription is applicable to the data. Unlike before, since we have security constraints in place, the subscription is not applicable:

>>> subscriptions = find_active_subscriptions_for(event.object, event)
>>> [subscription.isApplicable(event.object) for subscription in subscriptions]
[True]

Wait, wait…what happened there? It turns out that since we don’t have any defined principal identified by some.one, we use the global IUnauthenticatedPrincipal, an anonymous user. In turn, the directives executed by loading securitypolicy.zcml from zope.securitypolicy give anonymous users the zope.View permission by default. Let’s reverse that and check again.

>>> from zope.securitypolicy.rolepermission import rolePermissionManager
>>> rolePermissionManager.denyPermissionToRole('zope.View', 'zope.Anonymous')
>>> [subscription.isApplicable(event.object) for subscription in subscriptions]
[False]

Ahh, that’s better. We could have also disabled that behaviour.

Note

Dynamic (persistent) subscriptions do not fallback to the unauthenticated principal by default.

>>> subscriptions[0].fallback_to_unauthenticated_principal
True
>>> subscriptions[0].fallback_to_unauthenticated_principal = False

Subscription Applicable Once Principals are Defined¶

To grant access in an expected way, we’ll use zope.principalregistry to globally define the prinicpal we’re looking for, as well as globally grant that principal the permissions necessary:

>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.securitypolicy" file="meta.zcml" />
...   <include package="zope.principalregistry" file="meta.zcml" />
...   <principal
...         id="some.one"
...         title="Some One"
...         login="some.one"
...         password_manager="SHA1"
...         password="40bd001563085fc35165329ea1ff5c5ecbdbbeef"
...         />
...   <grant principal="some.one" permission="zope.View" />
... </configure>
... """)

Now our webhook is applicable:

>>> [subscription.isApplicable(event.object) for subscription in subscriptions]
[True]

Existing Interactions¶

If there was already an interaction going on (e.g., for the logged in user that created the object), the owner of the subscription is added to that interaction for purposes of checking permissions. Security policies generally only grant access if all participations in the interaction have access.

We’ll demonstrate this by creating and acting as a new principal and then checking access. Because our new user has no permissions on the object being created (which of course is highly unusual), the permission check will fail.

>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.principalregistry" file="meta.zcml" />
...   <principal
...         id="some.one.else"
...         title="Some One Else"
...         login="some.one.else"
...         password_manager="SHA1"
...         password="40bd001563085fc35165329ea1ff5c5ecbdbbeef"
...         />
... </configure>
... """)
>>> from zope.security.testing import interaction
>>> with interaction('some.one.else'):
...    [subscription.isApplicable(event.object) for subscription in subscriptions]
[False]

Automatic Deactivation On Failure¶

For any type of subscription (static, dynamic, persistent, …) that implements nti.webhooks.interfaces.ILimitedApplicabilityPreconditionFailureWebhookSubscription, attempting to make deliveries to it while it is misconfigured (e.g., there is no such permission defined or no principal can be found) will eventually result in it becoming automatically disabled.

Our subscription is such an object:

>>> from nti.webhooks.interfaces import ILimitedApplicabilityPreconditionFailureWebhookSubscription
>>> from zope.interface import verify
>>> subscription = subscriptions[0]
>>> verify.verifyObject(ILimitedApplicabilityPreconditionFailureWebhookSubscription, subscription)
True
>>> subscription.applicable_precondition_failure_limit
50
>>> subscription.active
True
>>> print(subscription.status_message)
Active

This doesn’t apply when the permission check is simply denied; that’s normal and expected.

>>> from delivery_helper import deliver_some
>>> from nti.webhooks.testing import wait_for_deliveries
>>> with interaction('some.one.else'):
...      deliver_some(100)
>>> len(subscription)
0
>>> subscription.active
True

But if we remove the principal our subscription is using (being careful not to fire any events that might automatically remove or deactivate the subscription) we can see that it becomes inactive at the correct time.

First, we’ll deliver one, just to prove it works.

>>> deliver_some()
>>> wait_for_deliveries()
>>> len(subscription)
1

Now we’ll destroy the principal registration, making this object incapable of accepting deliveries. (This works because we disabled the fallback to the unauthenticated principal earlier.)

>>> from zope.principalregistry import principalregistry
>>> principalregistry.principalRegistry._clear()

When we attempt enough of them, it is deactivated.

>>> deliver_some(49)
>>> len(subscription)
1
>>> subscription.active
True
>>> deliver_some(2)
>>> len(subscription)
1
>>> subscription.active
False
>>> print(subscription.status_message)
Delivery suspended due to too many precondition failures.

Manually activating the subscription resets the counter.

>>> subscription.__parent__.activateSubscription(subscription)
True
>>> subscription.active
True
>>> print(subscription.status_message)
Active
>>> deliver_some(50)
>>> subscription.active
False

Delivery History¶

All attempts at delivering a webhook are recorded as an object that implements IWebhookDeliveryAttempt. For static subscriptions, these are only in memory on a single machine, but persistent, dynamic, subscriptions that record their history more durably are also possible.

Delivery always occurs as a result of committing a transaction, and the resulting attempt object is stored in the corresponding subscription object.

Types of Delivery Attempts¶

There are three types of delivery attempts: pending, unsuccessful, and successful. A pending attempt is one that hasn’t yet been resolved, while the other two have been resolved.

Successful Delivery Attempts¶

Successful delivery attempts are the most interesting. Lets look at one of those and describe the IWebhookDeliveryAttempt. We begin by defining a subscription.

>>> from zope.configuration import xmlconfig
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.component" />
...   <include package="zope.container" />
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <webhooks:staticSubscription
...             to="https://example.com/some/path"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectCreatedEvent" />
... </configure>
... """)

Now we can access the subscription. Because delivery attempts are stored in the subscription, it has a length of 0 at this time.

>>> from zope import component
>>> from nti.webhooks import interfaces
>>> sub_manager = component.getUtility(interfaces.IWebhookSubscriptionManager)
>>> subscription = sub_manager['Subscription']
>>> len(subscription)
0

To avoid actually trying to talk to example.com, we’ll be using some mocks.

>>> from nti.webhooks.testing import mock_delivery_to
>>> mock_delivery_to('https://example.com/some/path', method='POST', status=200)

Now we will create the object, broadcast the event to engage the subscription, and commit the transaction to send the hook. A helper to do that is already defined.

import transaction
from zope import lifecycleevent
from zope import interface
from zope.container.folder import Folder
from zope.securitypolicy.interfaces import IPrincipalPermissionManager
from zope.annotation.interfaces import IAttributeAnnotatable

from nti.testing.time import time_monotonically_increases

from nti.webhooks.testing import wait_for_deliveries

__all__ = [
    'deliver_some',
    'wait_for_deliveries',
]

@time_monotonically_increases
def deliver_some(how_many=1, note=None, grants=None, event='created'):
    for _ in range(how_many):
        tx = transaction.begin()
        if note:
            tx.note(note)
        content = Folder()
        if grants:
            # Make sure we can use the default (annotatable)
            # permission managers.
            interface.alsoProvides(content, IAttributeAnnotatable)
            prin_perm = IPrincipalPermissionManager(content)
            for principal_id, perm_id in grants.items():
                prin_perm.grantPermissionToPrincipal(perm_id, principal_id)
        sender = getattr(lifecycleevent, event)
        sender(content)
        transaction.commit()

>>> from delivery_helper import deliver_some
>>> deliver_some(note=u'/some/request/path')

In the background, the IWebhookDeliveryManager is busy invoking the hook. We need to wait for it to finish, and then we can examine our delivery attempt:

>>> from delivery_helper import wait_for_deliveries
>>> wait_for_deliveries()

Attempt Details¶

The subscription now has an attempt recorded in the form of an IWebhookDeliveryAttempt. The attempt records some basic details, such as the overall status, and an associated message.

Important

Attempts are immutable. They are created and managed entirely by the system and mutation attempts are not allowed.

>>> len(subscription)
1
>>> attempt = subscription.pop()
>>> from zope.interface import verify
>>> verify.verifyObject(interfaces.IWebhookDeliveryAttempt, attempt)
True
>>> attempt.status
'successful'
>>> print(attempt.message)
200 OK

Also attached to the attempt is some debugging information. This information is intended for internal use and is not externalized. The exact details may change over time, but some information is always present.

>>> internal_info = attempt.internal_info
>>> internal_info.originated
DeliveryOriginationInfo(pid=..., hostname=..., createdTime=..., transaction_note=...'/some/request/path')
>>> internal_info.exception_history
()

An important attribute of the attempt is the request; this attribute (an IWebhookDeliveryAttemptRequest) provides information about the HTTP request as it went on the wire.

>>> verify.verifyObject(interfaces.IWebhookDeliveryAttemptRequest, attempt.request)
True
>>> print(attempt.request.url)
https://example.com/some/path
>>> print(attempt.request.method)
POST
>>> import pprint
>>> pprint.pprint({str(k): str(v) for k, v in attempt.request.headers.items()})
{'Accept': '*/*',
 'Accept-Encoding': 'gzip, deflate',
 'Connection': 'keep-alive',
 'Content-Length': '94',
 'Content-Type': 'application/json',
 'User-Agent': 'nti.webhooks ...'}
>>> print(attempt.request.body)
{"Class": "NonExternalizableObject", "InternalType": "<class 'zope.container.folder.Folder'>"}

(If you’re curious about that “NonExternalizableObject” business, then see Customizing HTTP Requests.)

Another important attribute is the response (an IWebhookDeliveryAttemptResponse), which captures information about the data received from the target.

>>> verify.verifyObject(interfaces.IWebhookDeliveryAttemptResponse, attempt.response)
True
>>> attempt.response.status_code
200
>>> print(attempt.response.reason)
OK
>>> pprint.pprint({str(k): str(v) for k, v in attempt.response.headers.items()})
{'Content-Type': 'text/plain'}
>>> print(attempt.response.content)

>>> attempt.response.elapsed
datetime.timedelta(...)

Failed Delivery Attempts¶

A delivery attempt fails when:

The subscription was active; and
The subscription was applicable; and
Some error occurred communicating with target (such errors include, but are not limited to, failed DNS lookups and HTTP error responses); OR
Some error occurred handling the response from the target; note that in this case, the target might have processed things correctly.

It has the same request and response attributes as successful attempts, but, depending on when the error occurred, one or both of them may be None. Details on the reasons for the failure may be found in the internal_info.exception_history, if the HTTP request wasn’t able to complete.

We’ll simulate some of these conditions using mocks. First, a failure communicating with the remote server.

>>> from nti.webhooks.testing import http_requests_fail
>>> with http_requests_fail():
...     deliver_some(note=u'this should fail remotely')
...     wait_for_deliveries()
>>> len(subscription)
1
>>> attempt = subscription.pop()
>>> attempt.status
'failed'
>>> print(attempt.message)
Contacting the remote server experienced an unexpected error.
>>> internal_info = attempt.internal_info
>>> print(internal_info.originated.transaction_note)
this should fail remotely
>>> len(internal_info.exception_history)
1
>>> print(internal_info.exception_history[0])
Traceback (most recent call last):
  Module nti.webhooks.delivery_manager...
...
...RequestException

Next, a failure to process the response.

>>> from nti.webhooks.testing import processing_results_fail
>>> with processing_results_fail():
...     deliver_some(note=u'this should fail locally')
...     wait_for_deliveries()
>>> len(subscription)
1
>>> attempt = subscription.pop()
>>> attempt.status
'failed'
>>> print(attempt.message)
Unexpected error handling the response from the server.
>>> internal_info = attempt.internal_info
>>> print(internal_info.originated.transaction_note)
this should fail locally
>>> len(internal_info.exception_history)
1
>>> print(internal_info.exception_history[0])
Traceback (most recent call last):
  Module nti.webhooks.delivery_manager...
...
UnicodeError

Pending Delivery Attempts¶

Pending delivery attempts are those scheduled for delivery (typically). The request and response attributes will always be None.

Limits On History¶

Only a limited number of delivery attempts are stored for any given subscription. Currently, a class (or instance!) attribute establishes this limit, but in the future this may be changed to something more flexible. In the future there may also be a limit of some sort per-principal.

>>> from nti.webhooks.interfaces import ILimitedAttemptWebhookSubscription
>>> from zope.interface import verify
>>> verify.verifyObject(ILimitedAttemptWebhookSubscription, subscription)
True
>>> subscription.attempt_limit
50
>>> len(subscription)
0

The limit doesn’t apply to pending attempts, only to successful or failed attempts. We can demonstrate this by switching to deferred delivery (meaning all attempts stay pending until we wait for them), creating a bunch of attempts, and looking at the length.

>>> from nti.webhooks.testing import begin_deferred_delivery
>>> begin_deferred_delivery()
>>> deliver_some(100)
>>> len(subscription)
100
>>> list(set(attempt.status for attempt in subscription.values()))
['pending']

They’re all pending. This is a good time to note that iterating the subscription does so in the order in which attempts were added, so the oldest attempt is first.

>>> list(subscription) == sorted(subscription)
True
>>> all_attempts = list(subscription.values())
>>> sorted_attempts = sorted(all_attempts, key=lambda attempt: attempt.createdTime)
>>> all_attempts == sorted_attempts
True
>>> oldest_attempt = all_attempts[0]
>>> attempt_50 = all_attempts[50]
>>> oldest_attempt.createdTime < attempt_50.createdTime < all_attempts[-1].createdTime
True

Now if we deliver them, the oldest pending attempts will be completed, and as the newer attempts complete, they will replace them.

>>> wait_for_deliveries()
>>> len(subscription)
50
>>> all_attempts = list(subscription.values())
>>> list(set(attempt.status for attempt in all_attempts))
['successful']
>>> oldest_attempt in all_attempts
False
>>> attempt_50 in all_attempts
True

Automatic Deactivation on Failures¶

If the history is completely filled up with failures, whether from validation errors, HTTP errors, or local processing errors, the subscription will be automatically marked inactive and future deliveries will not be attempted until it is manually activated again.

Note

At this time, pending deliveries are exempted from inactivating the subscription. This means a sudden large burst of pending deliveries could be scheduled and delivery attempted, even if all previous deliveries have failed.

HTTP Failures¶

Here, we’ll demonstrate this for HTTP failures.

>>> subscription.active
True
>>> print(subscription.status_message)
Active
>>> with http_requests_fail():
...     deliver_some(100, note=u'this should fail remotely')
...     wait_for_deliveries()
>>> len(subscription)
50
>>> all_attempts = list(subscription.values())
>>> list(set(attempt.status for attempt in all_attempts))
['failed']
>>> subscription.active
False
>>> print(subscription.status_message)
Delivery suspended due to too many delivery failures.

Attempting more deliveries of course doesn’t change this deactivated subscription in any way.

>>> deliver_some(100)
>>> wait_for_deliveries()
>>> len(subscription)
50
>>> all_attempts == list(subscription.values())
True

Local Failures¶

Next, we’ll demonstrate the same thing for processing failures. We must first clear and re-enable the subscription.

>>> def resetSubscription():
...     sub_manager.activateSubscription(subscription)
...     print(subscription.active)
...     print(subscription.status_message)
...     subscription.clear()
>>> resetSubscription()
True
Active

With that out of the way, we can simulate processing failures, and show the same outcome as for HTTP failures.

>>> with processing_results_fail():
...     deliver_some(100, note=u'this should fail locally')
...     wait_for_deliveries()
>>> len(subscription)
50
>>> all_attempts = list(subscription.values())
>>> list(set(attempt.status for attempt in all_attempts))
['failed']
>>> subscription.active
False
>>> print(subscription.status_message)
Delivery suspended due to too many delivery failures.
>>> deliver_some(100)
>>> wait_for_deliveries()
>>> len(subscription)
50
>>> all_attempts == list(subscription.values())
True

Validation Failures¶

Finally, the same results occur for validation failures.

>>> resetSubscription()
True
Active
>>> from nti.webhooks.testing import target_validation_fails
>>> with target_validation_fails():
...     deliver_some(100, note=u'this should fail validation')
...     wait_for_deliveries()
>>> len(subscription)
50
>>> all_attempts = list(subscription.values())
>>> list(set(attempt.status for attempt in all_attempts))
['failed']
>>> subscription.active
False
>>> print(subscription.status_message)
Delivery suspended due to too many delivery failures.
>>> deliver_some(100)
>>> wait_for_deliveries()
>>> len(subscription)
50
>>> all_attempts == list(subscription.values())
True

Todo

Similar tests for repeatedly inapplicable subscriptions.

Subscription Security¶

In Delivery Security Checks, we covered the checks that are applied when determining whether to deliver an event’s object to a particular subscription’s target.

This document will cover the security that’s applied to the subscriptions themselves.

Subscription security is based on zope.securitypolicy, using its concepts of principals, roles, permissions, and granting or denying permissions to principals or roles,

Each subscription that has an owner grants read (zope.View) and delete (nti.actions.delete) access to that owner. No other permissions are defined or used; any other access must be inherited from the subscription’s lineage. Typically this will mean that principals with the role zope.Manager will have complete access; if you load the securitypolicy.zcml configuration from zope.securitypolicy, then anonymous users will also have access. If there is no owner, then no specific grants will be made.

This applies equally for both static and dynamic subscriptions, and can be customized; see Customizing The Permission Grants for more information.

Set Up¶

The following documentation will use the basic set up presented here. We’ll begin by loading the necessary ZCML configuration, including the default Zope security policy (but we’ll disable the anonymous access for demonstration purposes).

>>> from zope.configuration import xmlconfig
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.component" />
...   <include package="zope.container" />
...   <include package="zope.principalregistry" />
...   <include package="zope.securitypolicy" />
...   <!-- This defines permission nti.actions.delete and must be
...        before zope.securitypolicy:securitypolicy.zcml
...        for the zope.Manager role to be granted it. -->
...   <include package="nti.webhooks" />
...   <include package="zope.securitypolicy" file="securitypolicy.zcml" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <deny role="zope.Anonymous" permission="zope.View" />
... </configure>
... """)

Next, we’ll define a few principals (‘Alice’ and ‘Bob’), including one (‘Manager’) that we grant the zope.Manager role to.

>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.securitypolicy" file="meta.zcml" />
...   <include package="zope.principalregistry" file="meta.zcml" />
...   <principal
...         id="webhook.alice"
...         title="Alice"
...         login="alice"
...         password_manager="SHA1"
...         password="40bd001563085fc35165329ea1ff5c5ecbdbbeef"
...         />
...   <principal
...         id="webhook.bob"
...         title="Bob"
...         login="bob"
...         password_manager="SHA1"
...         password="40bd001563085fc35165329ea1ff5c5ecbdbbeef"
...         />
...   <principal
...         id="webhook.manager"
...         title="Manager"
...         login="manager"
...         password_manager="SHA1"
...         password="40bd001563085fc35165329ea1ff5c5ecbdbbeef"
...         />
...   <grant role="zope.Manager" principal="webhook.manager" />
... </configure>
... """)

Lastly, we’ll define a subscription for Alice.

>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" />
...   <webhooks:staticSubscription
...             to="https://example.com/some/path"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectCreatedEvent"
...             permission="zope.View"
...             owner="webhook.alice" />
... </configure>
... """)

Access To The Subscription¶

Todo

Write a subscriber and example for when the owner_id of a subscription changes. Currently that’s forbidden by the docs.

We’ll use a helper function to demonstrate access rights.

>>> from zope.security.testing import interaction
>>> from zope.security import checkPermission
>>> from collections import namedtuple
>>> Permissions = namedtuple("Permissions",
...                          ('view', 'delete', 'other'))
>>> def checkPermissions(principal_id, object):
...    def check(permission_id):
...        with interaction(principal_id):
...            return checkPermission(permission_id, object)
...    return Permissions(
...       check('zope.View'),
...       check('nti.actions.delete'),
...       check('zope.ManageContent'),
...    )

The Alice principal has view and delete access to the subscription. We’ll use a helper function to demonstrate this.

>>> from zope import component
>>> from nti.webhooks import interfaces
>>> sub_manager = component.getUtility(interfaces.IWebhookSubscriptionManager)
>>> subscription = sub_manager['Subscription']
>>> checkPermissions('webhook.alice', subscription)
Permissions(view=True, delete=True, other=False)

The manager also has view and delete access, plus a bunch of other inherited things:

>>> checkPermissions('webhook.manager', subscription)
Permissions(view=True, delete=True, other=True)

The other principal has no access:

>>> checkPermissions('webhook.bob', subscription)
Permissions(view=False, delete=False, other=False)

Access to Delivery Attempts¶

The same access rights flow down to delivery attempts contained within a subscription. A helper to fire the deliveries is already defined.

import transaction
from zope import lifecycleevent
from zope import interface
from zope.container.folder import Folder
from zope.securitypolicy.interfaces import IPrincipalPermissionManager
from zope.annotation.interfaces import IAttributeAnnotatable

from nti.testing.time import time_monotonically_increases

from nti.webhooks.testing import wait_for_deliveries

__all__ = [
    'deliver_some',
    'wait_for_deliveries',
]

@time_monotonically_increases
def deliver_some(how_many=1, note=None, grants=None, event='created'):
    for _ in range(how_many):
        tx = transaction.begin()
        if note:
            tx.note(note)
        content = Folder()
        if grants:
            # Make sure we can use the default (annotatable)
            # permission managers.
            interface.alsoProvides(content, IAttributeAnnotatable)
            prin_perm = IPrincipalPermissionManager(content)
            for principal_id, perm_id in grants.items():
                prin_perm.grantPermissionToPrincipal(perm_id, principal_id)
        sender = getattr(lifecycleevent, event)
        sender(content)
        transaction.commit()

To avoid actually trying to talk to example.com, we’ll be using some mocks.

>>> from nti.webhooks.testing import mock_delivery_to
>>> mock_delivery_to('https://example.com/some/path', method='POST', status=200)

We’ll perform the delivery as someone with high-privileges (the manager); we have to perform the delivery as someone because we need to be able to check permissions on the object.

>>> from delivery_helper import deliver_some, wait_for_deliveries
>>> with interaction('webhook.manager'):
...     deliver_some(how_many=1, grants={'webhook.alice': 'zope.View'})
>>> wait_for_deliveries()

Now we can check the access rights for the recorded delivery attempt. All three principals should have the same access to the attempt that they had to the subscription.

>>> attempt = subscription.pop()
>>> checkPermissions('webhook.alice', attempt)
Permissions(view=False, delete=False, other=False)
>>> checkPermissions('webhook.manager', attempt)
Permissions(view=True, delete=True, other=True)
>>> checkPermissions('webhook.bob', attempt)
Permissions(view=False, delete=False, other=False)

Wait, how come no one except the manager had any access? What happened to Alice? When we popped the attempt out of the subscription, it lost its __parent__ and disconnected from the lineage and hence the inherited flow of permissions. Let’s put it back and check again.

>>> attempt.__parent__ is None
True
>>> subscription['attempt'] = attempt
>>> checkPermissions('webhook.alice', attempt)
Permissions(view=True, delete=True, other=False)
>>> checkPermissions('webhook.manager', attempt)
Permissions(view=True, delete=True, other=True)
>>> checkPermissions('webhook.bob', attempt)
Permissions(view=False, delete=False, other=False)

Security Proxies¶

The global subscription manager (used here) is not registered with any permission. That means it does not have a security proxy wrapped around it, and thus, any user could potentially trigger deliveries. Likewise, the site-specific subscription managers (documented in Dynamic Webhook Subscriptions) also do not have security proxies created for them.

Traversal, however, is free to create security proxies. If you’re not using security proxies, then take the appropriate care to honor these permissions. For example, Pyramid view registrations should use these permissions before allowing access.

Customizing The Permission Grants¶

For information on customizing the permission grants, see nti.webhooks.subscribers.apply_security_to_subscription().

nti.webhooks.subscribers.apply_security_to_subscription(subscription, event)[source]

Set the permissions for the subscription when it is added to a container.

By default, only the owner_id of the subscription gets any permissions (and those permissions are zope.View and nti.actions.delete). If there is no owner, no permissions are added.

If you want to add additional permissions, simply add an additional subscriber. If you want to change or replace the default permissions, add an adapter for the subscription (in the current site) implementing IWebhookSubscriptionSecuritySetter; in that case you will be completely responsible for all security declarations.

Here’s a demonstration using a nti.webhooks.interfaces.IWebhookSubscriptionSecuritySetter that does nothing. When we create a new subscription for Bob, no one (other than the manager, who inherits access) has any access.

>>> from nti.webhooks.interfaces import IWebhookSubscriptionSecuritySetter
>>> from nti.webhooks.interfaces import IWebhookSubscription
>>> from zope.interface import implementer
>>> from nti.webhooks.subscriptions import resetGlobals
>>> resetGlobals()
>>> @component.adapter(IWebhookSubscription)
... @implementer(IWebhookSubscriptionSecuritySetter)
... class NoOpSetter(object):
...     def __init__(self, context, request=None):
...         pass
...     def __call__(self, context, event=None):
...         pass
>>> from zope import component
>>> component.provideAdapter(NoOpSetter)
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" />
...   <webhooks:staticSubscription
...             to="https://example.com/some/path"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectCreatedEvent"
...             permission="zope.View"
...             owner="webhook.bob" />
... </configure>
... """)
>>> subscription = sub_manager['Subscription']
>>> checkPermissions('webhook.alice', subscription)
Permissions(view=False, delete=False, other=False)
>>> checkPermissions('webhook.bob', subscription)
Permissions(view=False, delete=False, other=False)
>>> checkPermissions('webhook.manager', subscription)
Permissions(view=True, delete=True, other=True)

Customizing HTTP Requests¶

Once an active subscription matches and is applicable for a certain combination of object and event, eventually it’s time to create the actual HTTP request (body and headers) that will be delivered to the target URL.

This document will outline how that is done, and discuss how to customize that process. It will use static subscriptions to demonstrate, but these techniques are equally relevant for dynamic subscriptions.

Let’s begin by registering an example static subscription, and refresh our memory of what the HTTP request looks like by default. First, the imports and creation of the static subscription; we’ll use the objects defined in employees.py:

from persistent import Persistent
from zope.container.contained import Contained
from zope.site.folder import Folder
from zope.site import LocalSiteManager
from zope.annotation.interfaces import IAttributeAnnotatable
from zope.interface import implementer

class Employees(Folder):
    def __init__(self):
        Folder.__init__(self)
        self['employees'] = Folder()
        self.setSiteManager(LocalSiteManager(self))

class Department(Employees):
    pass

class Office(Employees):
    pass

@implementer(IAttributeAnnotatable)
class Employee(Contained, Persistent):
    COUNTER = 0

    def __init__(self):
        self.__counter__ = self.COUNTER
        Employee.COUNTER += 1

    def __repr__(self):
        return "<Employee %s %d>" % (
            self.__name__,
            self.__counter__,
        )

class ExternalizableEmployee(Employee):
    def toExternalObject(self, **kwargs):
        return self.__name__


from zope.testing import cleanup
cleanup.addCleanUp(lambda: setattr(Employee, 'COUNTER', 0))

>>> import transaction
>>> from zope import lifecycleevent, component
>>> from zope.container.folder import Folder
>>> from employees import Employee
>>> from zope.configuration import xmlconfig
>>> from nti.webhooks.interfaces import IWebhookSubscriptionManager
>>> from nti.webhooks.interfaces import IWebhookDeliveryManager
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.component" />
...   <include package="zope.container" />
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <webhooks:staticSubscription
...             to="https://example.com/some/path"
...             for="employees.Employee"
...             when="zope.lifecycleevent.interfaces.IObjectCreatedEvent" />
... </configure>
... """)
>>> from nti.webhooks.testing import mock_delivery_to
>>> mock_delivery_to('https://example.com/some/path')

Next, we trigger the subscription and wait for it to be delivered.

>>> def trigger_delivery(factory=Employee, name=u'Bob', last_modified=None):
...    _ = transaction.begin()
...    employee = factory()
...    employee.__name__ = name
...    if last_modified: employee.LastModified = last_modified
...    lifecycleevent.created(employee)
...    transaction.commit()
...    component.getUtility(IWebhookDeliveryManager).waitForPendingDeliveries()
>>> trigger_delivery()

Finally, we can look at what we actually sent. It’s not too pretty.

>>> sub_manager = component.getUtility(IWebhookSubscriptionManager)
>>> subscription = sub_manager['Subscription']
>>> attempt = subscription.pop()
>>> attempt.response.status_code
200
>>> print(attempt.request.url)
https://example.com/some/path
>>> print(attempt.request.method)
POST
>>> import pprint
>>> pprint.pprint({str(k): str(v) for k, v in attempt.request.headers.items()})
{'Accept': '*/*',
 'Accept-Encoding': 'gzip, deflate',
 'Connection': 'keep-alive',
 'Content-Length': '84',
 'Content-Type': 'application/json',
 'User-Agent': 'nti.webhooks...'}
>>> print(attempt.request.body)
{"Class": "NonExternalizableObject", "InternalType": "<class 'employees.Employee'>"}

Customizing The Body¶

There are a few different ways to customize the body, and they can be applied at the same time.

The first way to customize the body is to register an adapter producing an IWebhookPayload. The adapter can be an adapter for just the object, or it can be a multi-adapter from the object and the event that triggered the subscription. By default, both an adapter named DefaultWebhookDialect.externalizer_name and the unnamed adapter are attempted. When such an adapter is found, its value is externalized instead of the target of the event.

Note

While a single adapter is frequently enough, multi-adapters are allowed in case the context of the event matters. For example, one might wish to externalize something different when an object is created versus when it is modified or deleted.

Important

The security checks described in Delivery Security Checks apply to the object of the triggering event, not the adapted value.

Single Adapters¶

Working from lowest priority to highest priority, let’s demonstrate some adapters.

First, an adapter for a single object with no name.

>>> from zope.interface import implementer
>>> from zope.component import adapter
>>> from nti.webhooks.interfaces import IWebhookPayload
>>> @implementer(IWebhookPayload)
... @adapter(Employee)
... def single_adapter(employee):
...    return employee.__name__
>>> component.provideAdapter(single_adapter)

Triggering the event now produces a different body.

>>> trigger_delivery()
>>> attempt = subscription.pop()
>>> print(attempt.request.body)
"Bob"
>>> trigger_delivery(name=u'Susan')
>>> attempt = subscription.pop()
>>> print(attempt.request.body)
"Susan"

Higher priority is a named adapter.

>>> from zope.component import named
>>> @implementer(IWebhookPayload)
... @adapter(Employee)
... @named("webhook-delivery")
... def named_single_adapter(folder):
...     return "An Employee"
>>> component.provideAdapter(named_single_adapter)
>>> trigger_delivery()
>>> attempt = subscription.pop()
>>> print(attempt.request.body)
"An Employee"

Of course, if the object already provides IWebhookPayload, then it is returned directly without using those adapters.

>>> @implementer(IWebhookPayload)
... class PayloadFactory(Employee):
...    """An employee that is its own payload."""
>>> trigger_delivery(factory=PayloadFactory)
>>> attempt = subscription.pop()
>>> print(attempt.request.body)
{"Class": "NonExternalizableObject", "InternalType": "<class 'PayloadFactory'>"}

Multi-adapters¶

Multi-adapters are the highest priority. They take precedence over the object itself being a IWebhookPayload already.

The unnamed adapter for the event and the object is higher priority than the named single adapter or the object itself.

>>> from zope.lifecycleevent.interfaces import IObjectCreatedEvent
>>> @implementer(IWebhookPayload)
... @adapter(Employee, IObjectCreatedEvent)
... def multi_adapter(employee, event):
...    return "employee-and-event"
>>> component.provideAdapter(multi_adapter)
>>> trigger_delivery()
>>> attempt = subscription.pop()
>>> print(attempt.request.body)
"employee-and-event"

Finally, the highest priority is a named multi-adapter.

>>> from zope.lifecycleevent.interfaces import IObjectCreatedEvent
>>> @implementer(IWebhookPayload)
... @adapter(Employee, IObjectCreatedEvent)
... @named("webhook-delivery")
... def named_multi_adapter(employee, event):
...    return "named-employee-and-event"
>>> component.provideAdapter(named_multi_adapter)
>>> trigger_delivery()
>>> attempt = subscription.pop()
>>> print(attempt.request.body)
"named-employee-and-event"

Cleanup¶

Let’s remove all those adapters and get back to a base state.

>>> gsm = component.getGlobalSiteManager()
>>> gsm.unregisterAdapter(named_multi_adapter, name=named_multi_adapter.__component_name__)
True
>>> gsm.unregisterAdapter(multi_adapter)
True
>>> gsm.unregisterAdapter(named_single_adapter, name=named_single_adapter.__component_name__)
True
>>> gsm.unregisterAdapter(single_adapter)
True

Webhook Dialects¶

Another way to customize the body, and much more, is to write a dialect. Every subscription is associated, by name, with a dialect. Dialects are registered utilities that implement nti.webhooks.interfaces.IWebhookDialect; there is a global default (the empty name, ‘’) dialect implemented in DefaultWebhookDialect. When defining new dialects, you should extend this class. In fact, the behaviour defined above is implemented by this class in its DefaultWebhookDialect.produce_payload() method.

Important

Dialects must not be persistent objects. They may be used outside of contexts where ZODB is available.

Note

Much of what is done next with custom code can also be done with ZCML. See Defining A Dialect Using ZCML for details.

Setting the Body¶

One easy way to customize the body is to use named externalizers. The default dialect uses an externalizer with the name given in externalizer_name; a subclass can change this by setting it on the class object. We’ll demonstrate by first defining and registering a IInternalObjectExternalizer with a custom name.

>>> from nti.externalization.interfaces import IInternalObjectExternalizer
>>> from nti.externalization import to_standard_external_dictionary
>>> @implementer(IInternalObjectExternalizer)
... @adapter(Employee)
... @named('webhook-testing')
... class EmployeeExternalizer(object):
...     def __init__(self, context):
...         self.context = context
...     def toExternalObject(self, **kwargs):
...         std = to_standard_external_dictionary(self.context, **kwargs)
...         std['Class'] = 'Employee'
...         std['Name'] = self.context.__name__
...         return std
>>> component.provideAdapter(EmployeeExternalizer)

Next, we’ll create a dialect that uses this externalizer, and register it:

>>> from nti.webhooks.dialect import DefaultWebhookDialect
>>> @named('webhook-testing')
... class TestDialect(DefaultWebhookDialect):
...     externalizer_name = 'webhook-testing'
>>> component.provideUtility(TestDialect())

We then alter the subscription to use this dialect:

>>> subscription.dialect_id = 'webhook-testing'

Now when we trigger the subscription, we use this externalizer:

>>> trigger_delivery()
>>> attempt = subscription.pop()
>>> print(attempt.request.body)
{"Class": "Employee", "Name": "Bob"}

Customizing the Body With Externalization Policies

Making smaller tweaks can be accomplished by adjusting the externalization policy that’s used. By default, the externalization policy, named in externalizer_policy_name, produces ISO8601 strings for values stored as Unix timestamps (seconds since the epoch).

>>> trigger_delivery(last_modified=123456789.0)
>>> attempt = subscription.pop()
>>> print(attempt.request.body)
{"Class": "Employee", "Last Modified": "1973-11-29T21:33:09Z", "Name": "Bob"}

The best way to adjust this is to set the externalizer_policy_name to a different value. A unicode string should refere to a registered externalization policy component. If we set it to None, the default policy (which outputs the timestamps as numbers) is used.

>>> TestDialect.externalizer_policy_name = None
>>> trigger_delivery(last_modified=123456789.0)
>>> attempt = subscription.pop()
>>> print(attempt.request.body)
{"Class": "Employee", "Last Modified": 123456789.0, "Name": "Bob"}

Setting Headers¶

The dialect is also responsible for customizing aspects of the HTTP request, including headers, authentication, and the method. Our previous attempt used the default values for these things:

>>> print(attempt.request.method)
POST
>>> import pprint
>>> pprint.pprint({str(k): str(v) for k, v in attempt.request.headers.items()})
{'Accept': '*/*',
 'Accept-Encoding': 'gzip, deflate',
 'Connection': 'keep-alive',
 'Content-Length': '66',
 'Content-Type': 'application/json',
 'User-Agent': 'nti.webhooks ...'}

Lets apply some simple customizations and send again.

>>> TestDialect.http_method = 'PUT'
>>> TestDialect.user_agent = 'doctests'
>>> trigger_delivery()
>>> attempt = subscription.pop()
>>> print(attempt.request.method)
PUT
>>> pprint.pprint({str(k): str(v) for k, v in attempt.request.headers.items()})
{'Accept': '*/*',
 'Accept-Encoding': 'gzip, deflate',
 'Connection': 'keep-alive',
 'Content-Length': '36',
 'Content-Type': 'application/json',
 'User-Agent': 'doctests'}

Defining A Dialect Using ZCML¶

For the simple cases that are customizations of the strings defined by the DefaultWebhookDialect, you can use a ZCML directive to define them.

interface nti.webhooks.zcml.IDialectDirective[source]

Create a new dialect subclass of DefaultWebhookDialect and register it as a utility named name.

name¶

Name

Name of the dialect registration. Limited to ASCII characters.

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

externalizer_name¶

The name of the externalization adapters to use

Remember, if adapters by this name do not exist, the default will be used.

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

externalizer_policy_name¶

The name of the externalizer policy component to use.

Important

An empty string selects the nti.externalization default policy, which uses Unix timestamps. To use the default policy of nti.webhooks, omit this argument.

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

http_method¶

The HTTP method to use.

This should be a valid method name, but that’s not enforced

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

user_agent¶

The User-Agent header string to use.

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

We can repeat the above example using just ZCML.

>>> from nti.webhooks.subscriptions import resetGlobals
>>> resetGlobals()
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.component" />
...   <include package="zope.container" />
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <webhooks:webhookDialect
...             name='zcml-dialect'
...             externalizer_name='webhook-testing'
...             externalizer_policy_name=''
...             http_method='PUT'
...             user_agent='zcml-tests' />
...   <webhooks:staticSubscription
...             dialect='zcml-dialect'
...             to="https://example.com/some/path"
...             for="employees.Employee"
...             when="zope.lifecycleevent.interfaces.IObjectCreatedEvent" />
... </configure>
... """)
>>> subscription = sub_manager['Subscription']
>>> trigger_delivery(last_modified=123456789.0)
>>> attempt = subscription.pop()
>>> print(attempt.request.body)
{"Class": "Employee", "Last Modified": 123456789.0, "Name": "Bob"}
>>> print(attempt.request.method)
PUT
>>> pprint.pprint({str(k): str(v) for k, v in attempt.request.headers.items()})
{'Accept': '*/*',
 'Accept-Encoding': 'gzip, deflate',
 'Connection': 'keep-alive',
 'Content-Length': '66',
 'Content-Type': 'application/json',
 'User-Agent': 'zcml-tests'}

Dynamic Webhook Subscriptions¶

In addition to static webhook subscriptions defined in ZCML, this package supports dynamic webhook subscriptions created, activated, inactivated, and removed through code at runtime. Such subscriptions, and their delivery history, are typically persistent in the ZODB sense of the word.

Subscriptions are managed via an implementation of nti.webhooks.interfaces.IWebhookSubscriptionManager. We’ve already seen (in Configured Global, Transient Webhook Subscriptions) how there is a global, non-persistent subscription manager installed by default. This document explores issues around persistent, local subscription managers.

Goals¶

Persistent subscription managers, and their subscriptions and in turn delivery histories, should:

Have complete paths, as defined by zope.location.interfaces.ILocationInfo.getPath() or zope.traversing.interfaces.ITraversalAPI.getPath().
Be fully traversable, as defined by zope.traversing.interfaces.ITraversalAPI.traverseName().

This second requirement is met by having the subscription manager be a zope.container.interfaces.IContainer of IWebhookSubscription objects, which in turn are containers of IWebhookDeliveryAttempt objects.

The first requirement is somewhat harder. This package offers a high-level API to help with it, integrated with nti.site. In this API, persistent webhook subscription managers are stored in the site manager using nti.site.localutility.install_utility() with a name in the etc namespace.

Sub-pages

This page has sub-pages for specific topics.

Customizing For¶

Sometimes the output of zope.interface.providedBy() is not what you want the high-level dynamic subscription APIs to use. This can be customized by providing an adapter for the object to nti.webhooks.interfaces.IWebhookResourceDiscriminator.

Example¶

Previously in Dynamic Webhook Subscriptions, we saw how employees.Employee got a very specific registration. Let’s add an adapter and see it get a different registration.

First, we create the normal configuration.

>>> from zope.configuration import xmlconfig
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <include package="nti.site" />
...   <include package="zope.traversing" />
... </configure>
... """)

Next, we create and provide an adapter. When called, it returns the specific class instead of its set of provided interfaces. Typically, an adapter will return one specific interface, but it can return anything that can be passed to zope.interface.interfaces.IComponentRegistry.registerAdapter() as part of the required parameter.

>>> from zope.interface import implementer
>>> from zope.component import adapter
>>> from zope.component import provideAdapter
>>> from employees import Employee
>>> from nti.webhooks.interfaces import IWebhookResourceDiscriminator
>>> @implementer(IWebhookResourceDiscriminator)
... @adapter(Employee)
... class EmployeeDiscriminator(object):
...     def __init__(self, context):
...         self.context = context
...     def __call__(self):
...         return type(self.context)
>>> provideAdapter(EmployeeDiscriminator)

Now, when we use the high-level API, we find a more specific for value:

>>> from nti.webhooks.api import subscribe_to_resource
>>> bob = Employee()
>>> subscription = subscribe_to_resource(bob, 'https://example.com/some/path')
>>> subscription.for_
<class 'employees.Employee'>

Setup¶

To begin, we will provide a persistent site hierarchy with traversable paths. Following the example from the main documentation, we’ll create a department named “NWS” and office named “OUN”, plus some people in each one. The department and office will be sites, with a site manager.

First define the classes. These are stored in a module named employees.

from persistent import Persistent
from zope.container.contained import Contained
from zope.site.folder import Folder
from zope.site import LocalSiteManager
from zope.annotation.interfaces import IAttributeAnnotatable
from zope.interface import implementer

class Employees(Folder):
    def __init__(self):
        Folder.__init__(self)
        self['employees'] = Folder()
        self.setSiteManager(LocalSiteManager(self))

class Department(Employees):
    pass

class Office(Employees):
    pass

@implementer(IAttributeAnnotatable)
class Employee(Contained, Persistent):
    COUNTER = 0

    def __init__(self):
        self.__counter__ = self.COUNTER
        Employee.COUNTER += 1

    def __repr__(self):
        return "<Employee %s %d>" % (
            self.__name__,
            self.__counter__,
        )

class ExternalizableEmployee(Employee):
    def toExternalObject(self, **kwargs):
        return self.__name__


from zope.testing import cleanup
cleanup.addCleanUp(lambda: setattr(Employee, 'COUNTER', 0))

>>> from employees import Department, Office, ExternalizableEmployee as Employee

Now we’ll create a database and store our hierarchy.

Note

The nti.webhooks.testing.ZODBFixture establishes a global, unnamed, utility for the ZODB.interfaces.IDatabase that it opens. This is what things like zope.app.appsetup do as well; your application needs to arrange for that utility to be available.

The nti.site.runner.run_job_in_site() function also has this requirement.

Begin with some common imports and set up the required packages and fixture.

>>> import transaction
>>> from nti.webhooks.testing import ZODBFixture
>>> from nti.webhooks.testing import DoctestTransaction
>>> from nti.webhooks.testing import mock_delivery_to
>>> from nti.site.hostpolicy import install_main_application_and_sites
>>> from nti.site.testing import print_tree
>>> from zope.traversing import api as ztapi
>>> from zope.configuration import xmlconfig
>>> mock_delivery_to('https://example.com/some/path', method='POST', status=200)
>>> mock_delivery_to('https://example.com/another/path', method='POST', status=404)
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <include package="nti.site" />
...   <include package="zope.traversing" />
... </configure>
... """)

Next, start a transaction and get a database connection, and add our objects. We can show that we have a traversable path to the lowest level object; we’ll use this path to refer to that object in the future (we don’t keep a reference to the actual object because we’ll be opening and closing multiple transactions).

>>> tx = DoctestTransaction()
>>> conn = tx.begin()
>>> root_folder, main_folder = install_main_application_and_sites(
...        conn,
...        root_alias=None, main_name='NOAA', main_alias=None)
>>> department = main_folder['NWS'] = Department()
>>> office = department['OUN'] = Office()
>>> department_bob = department['employees']['Bob'] = Employee()
>>> office_bob = office['employees']['Bob'] = Employee()
>>> print_tree(root_folder, depth=0, details=())
<ISite,IRootFolder>: <zope.site.folder.Folder object...>
     <ISite,IMainApplicationFolder>: NOAA
         ++etc++hostsites
         <ISite>: NWS
             <ISite>: OUN
                 employees
                     Bob => <Employee Bob 1>
             employees
                 Bob => <Employee Bob 0>
>>> office_bob_path = ztapi.getPath(office_bob)
>>> print(office_bob_path)
/NOAA/NWS/OUN/employees/Bob
>>> tx.finish()

High-level API¶

The high-level API lets us create subscriptions based on a resource, frequently one we’ve traversed to.

nti.webhooks.api.subscribe_to_resource(resource, to, for_=None, when=<InterfaceClass zope.interface.interfaces.IObjectEvent>, dialect_id=None, owner_id=None, permission_id=None)[source]

Produce and return a persistent IWebhookSubscription based on the resource.

Only the resource and to arguments are mandatory. The other arguments are optional, and are the same as the attributes in that interface.

Parameters:

resource –

The resource to subscribe to. Passing a resource does two things. First, the resources is used to find the closest enclosing ISite that is persistent. A IWebhookSubscriptionManager utility will be installed in this site if one does not already exist, and the subscription will be created there.

Second, if for_ is not given, then the interfaces provided by the resource will be used for for_. This doesn’t actually subscribe just to events on that exact object, but to events for objects with the same set of interfaces.

>>> from nti.webhooks.api import subscribe_to_resource
>>> conn = tx.begin()
>>> office_bob = ztapi.traverse(conn.root()['Application'], office_bob_path)
>>> subscription = subscribe_to_resource(office_bob, 'https://example.com/some/path')
>>> subscription
<...PersistentSubscription at 0x... to='https://example.com/some/path' for=employees.ExternalizableEmployee when=IObjectEvent>

What Just Happened¶

Several things happened here. The next sections will detail them.

A Subscription Manager Was Created¶

First, by getting the path to the subscription, we can see that a subscription manager containing the subscription was created at the closest enclosing site manager. We can also traverse this path to get back to the subscription, and its manager:

>>> path = ztapi.getPath(subscription)
>>> print(path)
/NOAA/NWS/OUN/++etc++site/default/WebhookSubscriptionManager/PersistentSubscription
>>> ztapi.traverse(root_folder, path) is subscription
True
>>> ztapi.traverse(root_folder, '/NOAA/NWS/OUN/++etc++site/default/WebhookSubscriptionManager')
<....PersistentWebhookSubscriptionManager object at 0x...>

The `for` Was Inferred¶

The API automatically deduced the value to use for for, in this case the same thing that office_bob provides:

>>> from zope.interface import providedBy
>>> subscription.for_
classImplements(ExternalizableEmployee)
>>> subscription.for_.__name__
'employees.ExternalizableEmployee'
>>> providedBy(office_bob)
classImplements(ExternalizableEmployee)
>>> providedBy(office_bob).inherit
<class 'employees.ExternalizableEmployee'>

This is a complex value; because of how pickling works, it will stay in sync with exactly what that class actually provides.

>>> list(providedBy(office_bob).flattened())
[<InterfaceClass ...IContained>, <InterfaceClass ...ILocation>, <InterfaceClass ...IPersistent>, <InterfaceClass ...Interface>]
>>> import pickle
>>> pickle.loads(pickle.dumps(subscription.for_)) is providedBy(office_bob)
True

For instructions on customizing how this is inferred, see Customizing For.

The `when` Was Guessed¶

Note

The value for when, IObjectEvent, may not be what you want. This may change in the future. See Configuration for more information.

Caution

This may change in the future. While it might be nice to have a single subscription that is when any of a group of events fires, the Zapier API prefers to have one subscription per event type. (TODO: Confirm this.) If that’s the case, then there might be a higher-level concept to group related subscriptions together.

The Subscription is Active¶

Even though the office that contains this subscription is not the current site, we can still find this subscription and confirm that it is active.

>>> def subscriptions_for_bob(conn):
...     from nti.webhooks.subscribers import find_active_subscriptions_for
...     from zope.lifecycleevent import ObjectModifiedEvent
...     office_bob = ztapi.traverse(conn.root()['Application'], office_bob_path)
...     event = ObjectModifiedEvent(office_bob)
...     return find_active_subscriptions_for(event.object, event)
>>> from zope.component import getSiteManager
>>> getSiteManager() is office.getSiteManager()
False
>>> len(subscriptions_for_bob(conn))
1
>>> subscriptions_for_bob(conn)[0] is subscription
True
>>> tx.finish()

Relation To Static Subscriptions¶

A static subscription registered globally is also found:

>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.component" />
...   <include package="zope.container" />
...   <include package="nti.webhooks" />
...   <webhooks:staticSubscription
...             to="https://example.com/another/path"
...             for="employees.Employee"
...             when="zope.lifecycleevent.interfaces.IObjectModifiedEvent" />
... </configure>
... """)
>>> conn = tx.begin()
>>> subscriptions = subscriptions_for_bob(conn)
>>> len(subscriptions)
2
>>> subscriptions
[<...Subscription at 0x... to='https://example.com/another/path' for=Employee when=IObjectModifiedEvent>, <...PersistentSubscription at 0x... to='https://example.com/some/path' ... when=IObjectEvent>]
>>> tx.finish()

Delivery to Static and Dynamic Subscriptions¶

Now we can attempt delivery to these subscriptions. They will have a delivery attempt recorded, and in the case of the persistent subscription, it will be persistent itself.

First, we define a helper function that will trigger and wait for the deliveries. We also ensure that the deliveries happen in a deterministic order.

>>> from nti.webhooks.testing import begin_synchronous_delivery
>>> begin_synchronous_delivery()
>>> def trigger_delivery():
...    from zope import lifecycleevent, component
...    from nti.webhooks.interfaces import IWebhookDeliveryManager
...    conn = tx.begin()
...    office_bob = ztapi.traverse(conn.root()['Application'], office_bob_path)
...    lifecycleevent.modified(office_bob)
...    tx.finish()
...    component.getUtility(IWebhookDeliveryManager).waitForPendingDeliveries()

Next, we deliver the events, and then fetch the updated subscriptions.

>>> trigger_delivery()
>>> conn = tx.begin()
>>> subscriptions = subscriptions_for_bob(conn)
>>> subscriptions
[<...Subscription at 0x... to='https://example.com/another/path' for=Employee when=IObjectModifiedEvent>, <...PersistentSubscription at 0x... to='https://example.com/some/path' ... when=IObjectEvent>]
>>> global_subscription = subscriptions[0]
>>> persistent_subscription = subscriptions[1]

Our attempt at persistent delivery was successful.

>>> attempt = persistent_subscription.pop()
>>> print(attempt.status)
successful
>>> attempt.response.status_code
200
>>> print(attempt.request.url)
https://example.com/some/path
>>> print(attempt.request.method)
POST
>>> import pprint
>>> pprint.pprint({str(k): str(v) for k, v in attempt.request.headers.items()})
{'Accept': '*/*',
 'Accept-Encoding': 'gzip, deflate',
 'Connection': 'keep-alive',
 'Content-Length': '5',
 'Content-Type': 'application/json',
 'User-Agent': 'nti.webhooks...'}
>>> print(attempt.request.body)
"Bob"
>>> tx.finish()

Because of the way the mock HTTP responses were set up, the static/global subscription delivery failed.

>>> attempt = global_subscription.pop()
>>> print(attempt.status)
failed
>>> print(attempt.message)
404 Not Found
>>> attempt.response.status_code
404
>>> print(attempt.request.url)
https://example.com/another/path
>>> print(attempt.request.method)
POST
>>> import pprint
>>> pprint.pprint({str(k): str(v) for k, v in attempt.request.headers.items()})
{'Accept': '*/*',
 'Accept-Encoding': 'gzip, deflate',
 'Connection': 'keep-alive',
 'Content-Length': '5',
 'Content-Type': 'application/json',
 'User-Agent': 'nti.webhooks...'}
>>> print(attempt.request.body)
"Bob"
>>> len(attempt.internal_info.exception_history)
0

Removing Subscriptions¶

After having created subscriptions through the API (see Dynamic Webhook Subscriptions), there are circumstances under which we may want to remove them. If we have the path to the subscription object, removing it is easy: Just remove it from its parent container (which can be obtained through traversal) as usual.

But there are other circumstances in which subscriptions should be removed. This document outlines some of them and the support that this package provides.

Principal Removal¶

When a subscription is owned by a particular principal, usually we’ll want to remove it when the principal itself is removed from the system.

To support this, this package provides a subscriber that handles removing subscriptions owned by a principal.

Important

Because this package can’t know for sure what the appropriate event is, it provides no ZCML to register this subscriber. You are responsible for making that registration.

We’ll demonstrate this by setting up a site tree similarly to how it was done in Dynamic Webhook Subscriptions.

First, the common imports and a ZODB database. This is the same as in Dynamic Webhook Subscriptions, except that we’re also configuring zope.pluggableauth because we’ll use that to be our principal implementation (in turn, that needs zope.password); configuring zope.securitypolicy is needed here because, unlike in that document, we’ll be specifying subscription owners and we need the adapters to be able to configure the security settings for those objects.

>>> from employees import Department, Office, ExternalizableEmployee as Employee
>>> import transaction
>>> from nti.webhooks.testing import DoctestTransaction
>>> from nti.webhooks.testing import mock_delivery_to
>>> from nti.site.hostpolicy import install_main_application_and_sites
>>> from nti.site.testing import print_tree
>>> from zope.traversing import api as ztapi
>>> from zope.configuration import xmlconfig
>>> mock_delivery_to('https://example.com/some/path', method='POST', status=200)
>>> mock_delivery_to('https://example.com/another/path', method='POST', status=404)
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <include package="nti.site" />
...   <include package="zope.traversing" />
...   <include package="zope.pluggableauth" />
...   <include package="zope.pluggableauth.plugins" file="principalfolder.zcml" />
...   <include package="zope.password" />
...   <include package="zope.securitypolicy" />
...   <include package="zope.securitypolicy" file="securitypolicy.zcml" />
... </configure>
... """)

Next, we duplicate our site setup, including creating two employee objects.

>>> tx = DoctestTransaction()
>>> conn = tx.begin()
>>> root_folder, main_folder = install_main_application_and_sites(
...        conn,
...        root_alias=None, main_name='NOAA', main_alias=None)
>>> department = main_folder['NWS'] = Department()
>>> office = department['OUN'] = Office()
>>> department_bob = department['employees']['Bob'] = Employee()
>>> office_bob = office['employees']['Bob'] = Employee()
>>> print_tree(root_folder, depth=0, details=())
<ISite,IRootFolder>: <zope.site.folder.Folder object...>
     <ISite,IMainApplicationFolder>: NOAA
         ++etc++hostsites
         <ISite>: NWS
             <ISite>: OUN
                 employees
                     Bob => <Employee Bob 1>
             employees
                 Bob => <Employee Bob 0>

We’ll then create corresponding principals for these two employees using zope.pluggableauth.plugins.principalfolder.

>>> from zope.authentication.interfaces import IAuthentication
>>> from zope.pluggableauth.interfaces import IAuthenticatorPlugin
>>> from zope.pluggableauth.authentication import PluggableAuthentication
>>> from zope.pluggableauth.plugins.principalfolder import PrincipalFolder
>>> from zope.pluggableauth.plugins.principalfolder import InternalPrincipal
>>> dep_auth = department.getSiteManager()['default']['authentication'] = PluggableAuthentication()
>>> department.getSiteManager().registerUtility(dep_auth, IAuthentication)
>>> nws_principals = PrincipalFolder('nws.')
>>> dbob_prin = nws_principals['bob'] = InternalPrincipal('login', 'password', 'title')
>>> dep_auth['principals'] = nws_principals
>>> dep_auth.authenticatorPlugins = ('principals',)
>>> office_auth = office.getSiteManager()['default']['authentication'] = PluggableAuthentication()
>>> office.getSiteManager().registerUtility(office_auth, IAuthentication)
>>> office_principals = PrincipalFolder('nws.oun.')
>>> obob_prin = office_principals['bob'] = InternalPrincipal('login', 'password', 'title')
>>> office_auth['principals'] = office_principals
>>> office_auth.authenticatorPlugins = ('principals',)
>>> print_tree(root_folder, depth=0, details=('siteManager',))
<ISite,IRootFolder>: <zope.site.folder.Folder ...>
     <ISite,IMainApplicationFolder>: NOAA
         ++etc++hostsites
         <ISite>: NWS
             <ISite>: OUN
                 employees
                     Bob => <Employee Bob 1>
                 <Site Manager> name=++etc++site
                     default
                         authentication
                             principals
                                 bob => <....InternalPrincipal object ...>
             employees
                 Bob => <Employee Bob 0>
             <Site Manager> name=++etc++site
                 default
                     authentication
                         principals
                             bob => <...InternalPrincipal object ...>
         <Site Manager> name=++etc++site
             default
     <Site Manager> name=++etc++site
         default

The lowest level principal folder can resolve both principals, but the higher level can resolve only the one defined at that level. Note how prefixes have been attached to the principal IDs.

>>> office_auth.getPrincipal('nws.oun.bob')
Principal('nws.oun.bob')
>>> office_auth.getPrincipal('nws.bob')
Principal('nws.bob')
>>> dep_auth.getPrincipal('nws.bob')
Principal('nws.bob')
>>> dep_auth.getPrincipal('nws.oun.bob')
Traceback (most recent call last):
...
zope.authentication.interfaces.PrincipalLookupError: oun.bob

Now that we have principals, with IDs, lets have them each subscribe to their own employee object, and commit the transaction.

>>> from nti.webhooks.api import subscribe_to_resource
>>> obob_sub = subscribe_to_resource(office_bob, 'https://example.com/some/path',
...    owner_id=u'nws.oun.bob', permission_id='zope.View')
>>> obob_sub
<...PersistentSubscription ... to='https://example.com/some/path' for=employees.ExternalizableEmployee when=IObjectEvent>
>>> dbob_sub = subscribe_to_resource(department_bob, 'https://example.com/another/path',
...    owner_id=u'nws.bob', permission_id='zope.View')
>>> dbob_sub
<...PersistentSubscription ... to='https://example.com/another/path' for=employees.ExternalizableEmployee when=IObjectEvent>
>>> print_tree(root_folder, depth=0, details=('siteManager'))
<ISite,IRootFolder>: <zope.site.folder.Folder ...>
     <ISite,IMainApplicationFolder>: NOAA
         ++etc++hostsites
         <ISite>: NWS
             <ISite>: OUN
                 employees
                     Bob => <Employee Bob 1>
                 <Site Manager> name=++etc++site
                     default
                         WebhookSubscriptionManager
                             PersistentSubscription
                         authentication
                             principals
                                 bob => ...
             employees
                 Bob => <Employee Bob 0>
             <Site Manager> name=++etc++site
                 default
                     WebhookSubscriptionManager
                         PersistentSubscription
                     authentication
                         principals
                             bob => ...
         <Site Manager> name=++etc++site
             default
     <Site Manager> name=++etc++site
         default
>>> tx.finish()

Lets deliver some hooks to both subscriptions in order to have something to look at.

>>> from nti.webhooks.testing import begin_synchronous_delivery
>>> begin_synchronous_delivery()
>>> def trigger_delivery():
...    from zope import lifecycleevent, component
...    from nti.webhooks.interfaces import IWebhookDeliveryManager
...    conn = tx.begin()
...    office_bob_path = '/NOAA/NWS/OUN/employees/Bob'
...    office_bob = ztapi.traverse(conn.root()['Application'], office_bob_path)
...    lifecycleevent.modified(office_bob)
...    tx.finish()
...    component.getUtility(IWebhookDeliveryManager).waitForPendingDeliveries()
>>> from zope.security.testing import interaction
>>> with interaction('nws.oun.bob'):
...     trigger_delivery()
>>> with interaction('nws.bob'):
...     trigger_delivery()

We used the office Bob as the context, so both subscriptions were found. And we specified zope.View as the permission, which by default is granted to everyone authenticated principal. Thus, both subscriptions have recorded two delivery attempts.

>>> _ = tx.begin()
>>> len(obob_sub)
2
>>> len(dbob_sub)
2
>>> tx.finish()

Registering The Handler¶

Lets register the handler and pretend to remove a principal. Hopefully the matching subscriptions are removed too.

nti.webhooks.subscribers.remove_subscriptions_for_principal(principal, event)[source]

Subscriber to find and remove all subscriptions for the principal when it is removed.

This is an adapter for (IPrincipal, IObjectRemovedEvent) by default, but that may not be the correct event in every system. Register it for the appropriate events in your system.

Parameters:

principal –
The principal being removed. It should still be located (having a proper __parent__) when this subscriber is invoked; this is the default for zope.container objects that use zope.container.contained.uncontained() in their __delitem__ method.

This can be any type of object. It is first adapted to nti.webhooks.interfaces.IWebhookPrincipal; if that fails, it is adapted to IPrincipal, and if that fails, it is used as-is. The final object must have the id attribute.
event – This is not used by this subscriber.

This subscriber removes all subscriptions owned by the principal found in subscription managers:

in the current site; and
in sites up the lineage of the original principal and adapted object (if different).

If the principal may have subscriptions in more places, provide an implementation of nti.webhooks.interfaces.IWebhookSubscriptionManagers for the original principal object. One (exhaustive) implementation is provided (but not registered) in ExhaustiveWebhookSubscriptionManagers.

>>> from nti.webhooks.subscribers import remove_subscriptions_for_principal
>>> from zope import component
>>> from zope.lifecycleevent import removed
>>> component.provideHandler(remove_subscriptions_for_principal)
>>> _ = tx.begin()
>>> removed(obob_prin)
>>> print_tree(department, depth=0, details=('siteManager',))
<ISite>: NWS
     <ISite>: OUN
         employees
             Bob => <Employee Bob 1>
         <Site Manager> name=++etc++site
             default
                 WebhookSubscriptionManager
                     PersistentSubscription
                         ... => <...PersistentWebhookDeliveryAttempt ... status='successful'>
                         ... => <...PersistentWebhookDeliveryAttempt ... status='successful'>
                 authentication
                     principals
                         bob => <....InternalPrincipal object ...>
     employees
         Bob => <Employee Bob 0>
     <Site Manager> name=++etc++site
         default
             WebhookSubscriptionManager
                 PersistentSubscription
                     ... => <...PersistentWebhookDeliveryAttempt ... status='failed'>
                     ... => <...PersistentWebhookDeliveryAttempt ... status='failed'>
             authentication
                 principals
                     bob => <....InternalPrincipal object ...>

They weren’t. In fact, the subscriber didn’t even run. Why not?

It turns out the InternalPrincipal objects don’t implement IPrincipal, so, as the docstring warned, the default registration wasn’t suitable here. We can fix that and try again.

>>> from zope.pluggableauth.plugins.principalfolder import IInternalPrincipal
>>> from zope.lifecycleevent.interfaces import IObjectRemovedEvent
>>> component.provideHandler(remove_subscriptions_for_principal,
...                          (IInternalPrincipal, IObjectRemovedEvent))
>>> removed(obob_prin)
Traceback (most recent call last):
...
AttributeError: 'InternalPrincipal' object has no attribute 'id'

Narf. Also as the docstring warned, the object being removed isn’t actually a IPrincipal and doesn’t have a compatible interface. We can fix that too! Since it should work this time, we’ll actually remove the principal.

>>> from nti.webhooks.interfaces import IWebhookPrincipal
>>> from zope.interface import implementer
>>> @implementer(IWebhookPrincipal)
... @component.adapter(IInternalPrincipal)
... class InternalPrincipalWebhookPrincipal(object):
...    def __init__(self, context):
...        self.id = context.__parent__.getIdByLogin(context.login)
>>> component.provideAdapter(InternalPrincipalWebhookPrincipal)
>>> del office_principals['bob']
>>> print_tree(department, depth=0, details=('siteManager',))
<ISite>: NWS
     <ISite>: OUN
         employees
             Bob => <Employee Bob 1>
         <Site Manager> name=++etc++site
             default
                 WebhookSubscriptionManager
                 authentication
                     principals
     employees
         Bob => <Employee Bob 0>
     <Site Manager> name=++etc++site
         default
             WebhookSubscriptionManager
                 PersistentSubscription
                     ... => <...PersistentWebhookDeliveryAttempt ... status='failed'>
                     ... => <...PersistentWebhookDeliveryAttempt ... status='failed'>
             authentication
                 principals
                     bob => <....InternalPrincipal object ...>
>>> tx.finish()

There! That did it.

Subscription Managers Outside The Site and Lineage¶

The documentation for remove_subscriptions_for_principal() mentions that only subscription managers in the current site hierarchy, and the hierarchy of the principal being removed, can be removed automatically. If there are subscriptions located outside this area, they won’t be removed. We can demonstrate this by setting up such a subscription. First, we need to add a new site; once that’s done, we can add the subscription and commit.

>>> from nti.webhooks.api import subscribe_in_site_manager
>>> _ = tx.begin()
>>> office = department['AMA'] = Office()
>>> subscribe_in_site_manager(office.getSiteManager(),
...    dict(to='https://example.com', for_=type(office_bob),
...         when=IObjectRemovedEvent, owner_id=u'nws.bob'))
<....PersistentSubscription ...>
>>> print_tree(department, depth=0, details=('siteManager',))
<ISite>: NWS
     <ISite>: AMA
         employees
         <Site Manager> name=++etc++site
             default
                 WebhookSubscriptionManager
                     PersistentSubscription
     <ISite>: OUN
         employees
             Bob => <Employee Bob 1>
         <Site Manager> name=++etc++site
             default
                 WebhookSubscriptionManager
                 authentication
                     principals
     employees
         Bob => <Employee Bob 0>
     <Site Manager> name=++etc++site
         default
             WebhookSubscriptionManager
                 PersistentSubscription
                     ... => <...PersistentWebhookDeliveryAttempt ... status='failed'>
                     ... => <...PersistentWebhookDeliveryAttempt ... status='failed'>
             authentication
                 principals
                     bob => <....InternalPrincipal object ...>
>>> tx.finish()

Now what happens when we delete “nws.bob”? That principal is above the subscription that was just created.

>>> _ = tx.begin()
>>> del nws_principals['bob']
>>> print_tree(department, depth=0, details=('siteManager',))
<ISite>: NWS
     <ISite>: AMA
         employees
         <Site Manager> name=++etc++site
             default
                 WebhookSubscriptionManager
                     PersistentSubscription
     <ISite>: OUN
         employees
             Bob => <Employee Bob 1>
         <Site Manager> name=++etc++site
             default
                 WebhookSubscriptionManager
                 authentication
                     principals
     employees
         Bob => <Employee Bob 0>
     <Site Manager> name=++etc++site
         default
             WebhookSubscriptionManager
             authentication
                 principals
>>> tx.abort()

We can see that we deleted the principal, and the subscription at the same level, but we didn’t find the unrelated subscription.

The solution to this is generally application specific. You can either listen for the event yourself, or register an appropriate nti.webhooks.interfaces.IWebhookSubscriptionManagers adapter. For simple, small, applications, the nti.webhooks.subscribers.ExhaustiveWebhookSubscriptionManagers can be used.

>>> from nti.webhooks.subscribers import ExhaustiveWebhookSubscriptionManagers
>>> component.provideAdapter(ExhaustiveWebhookSubscriptionManagers, (IInternalPrincipal,))
>>> _ = tx.begin()
>>> del nws_principals['bob']
>>> print_tree(department, depth=0, details=('siteManager',))
<ISite>: NWS
     <ISite>: AMA
         employees
         <Site Manager> name=++etc++site
             default
                 WebhookSubscriptionManager
     <ISite>: OUN
         employees
             Bob => <Employee Bob 1>
         <Site Manager> name=++etc++site
             default
                 WebhookSubscriptionManager
                 authentication
                     principals
     employees
         Bob => <Employee Bob 0>
     <Site Manager> name=++etc++site
         default
             WebhookSubscriptionManager
             authentication
                 principals
>>> tx.finish()

Events¶

This document describes the events that are fired specific to this package.

TODO: Do that.

Externalization¶

The subscription object and the items it contains (delivery attempts and their requests and responses) can be externalized using nti.externalization.

Note

Subscription managers are not currently defined for externalization.

Note

This is uni-directional. They can be externalized, but there is no direct support for internalizing them (updating them from external data). Conceptually, the delivery attempt and what it contains is immutable. Other than changing its active status (which is handled via other means) there is no use-case for mutating a subscription at this time.

Externalizing a subscription externalizes all contained delivery attempts. Since there is a strict limit on the number of attempts it can contain, this is not expected to pose a practical problem.

Externalizing a Subscription¶

Let’s see what it looks like when we externalize a subscription.

First, we define the subscription.

>>> from zope.configuration import xmlconfig
>>> conf_context = xmlconfig.string("""
... <configure
...     xmlns="http://namespaces.zope.org/zope"
...     xmlns:webhooks="http://nextthought.com/ntp/webhooks"
...     >
...   <include package="zope.component" />
...   <include package="zope.container" />
...   <include package="nti.webhooks" />
...   <include package="nti.webhooks" file="subscribers_promiscuous.zcml" />
...   <webhooks:staticSubscription
...             to="https://example.com/some/path"
...             for="zope.container.interfaces.IContentContainer"
...             when="zope.lifecycleevent.interfaces.IObjectCreatedEvent" />
... </configure>
... """)
>>> from zope import component
>>> from nti.webhooks import interfaces
>>> sub_manager = component.getUtility(interfaces.IWebhookSubscriptionManager)
>>> subscription = sub_manager['Subscription']

We’ll fill in some interesting mock data by making a few delivery attempts, one successful and one unsuccessful. To make it slightly easier to read, we’ll provide a simple payload adapter to customize the request body.

>>> from nti.webhooks.interfaces import IWebhookPayload
>>> from zope.interface import implementer
>>> from zope.component import adapter
>>> @implementer(IWebhookPayload)
... @adapter(object)
... def single_adapter(employee):
...    return "PAYLOAD"
>>> component.provideAdapter(single_adapter)

Ok, now we can make the deliveries.

>>> from nti.webhooks.testing import mock_delivery_to
>>> from nti.webhooks.testing import begin_synchronous_delivery
>>> begin_synchronous_delivery()
>>> from delivery_helper import deliver_some
>>> from delivery_helper import wait_for_deliveries
>>> mock_delivery_to('https://example.com/some/path', method='POST', status=200)
>>> mock_delivery_to('https://example.com/some/path', method='POST', status=404)
>>> deliver_some(note=u'/some/request/path')
>>> deliver_some(note=u'/another/request/path')
>>> wait_for_deliveries()

Externalizing the subscription now produces some useful data.

>>> from nti.externalization import to_external_object
>>> from pprint import pprint
>>> ext_subscription = to_external_object(subscription)

To make it easier to digest, we’ll look at the component objects one at a time. First, we’ll look at the subscription.

>>> ext_delivery_attempts = ext_subscription.pop('Contents')
>>> pprint(ext_subscription)
{'Class': 'Subscription',
 'CreatedTime': ...,
 'Last Modified': ...,
 'MimeType': 'application/vnd.nextthought.webhooks.webhooksubscription',
 'active': True,
 'attempt_limit': 50,
 'dialect_id': None,
 'for_': 'IContentContainer',
 'owner_id': None,
 'permission_id': None,
 'status_message': 'Active',
 'to': 'https://example.com/some/path',
 'when': 'IObjectCreatedEvent'}

Then the successful attempt:

>>> pprint(ext_delivery_attempts[0])
{'Class': 'WebhookDeliveryAttempt',
 'CreatedTime': ...,
 'Last Modified': ...,
 'MimeType': 'application/vnd.nextthought.webhooks.webhookdeliveryattempt',
 'message': '200 OK',
 'request': {'Class': 'WebhookDeliveryAttemptRequest',
             'CreatedTime': ...,
             'Last Modified': ...,
             'MimeType': 'application/vnd.nextthought.webhooks.webhookdeliveryattemptrequest',
             'body': '"PAYLOAD"',
             'headers': {'Accept': '*/*',
                         'Accept-Encoding': 'gzip, deflate',
                         'Connection': 'keep-alive',
                         'Content-Length': '9',
                         'Content-Type': 'application/json',
                         'User-Agent': 'nti.webhooks...'},
             'method': 'POST',
             'url': 'https://example.com/some/path'},
 'response': {'Class': 'WebhookDeliveryAttemptResponse',
              'CreatedTime': ...,
              'Last Modified': ...,
              'MimeType': 'application/vnd.nextthought.webhooks.webhookdeliveryattemptresponse',
              'content': '',
              'elapsed': 'PT0...S',
              'headers': {'Content-Type': 'text/plain'},
              'reason': 'OK',
              'status_code': 200},
 'status': 'successful'}

Followed by the failed attempt:

>>> pprint(ext_delivery_attempts[1])
{'Class': 'WebhookDeliveryAttempt',
 'CreatedTime': ...,
 'Last Modified': ...,
 'MimeType': 'application/vnd.nextthought.webhooks.webhookdeliveryattempt',
 'message': '404 Not Found',
 'request': {'Class': 'WebhookDeliveryAttemptRequest',
             'CreatedTime': ...,
             'Last Modified': ...,
             'MimeType': 'application/vnd.nextthought.webhooks.webhookdeliveryattemptrequest',
             'body': '"PAYLOAD"',
             'headers': {'Accept': '*/*',
                         'Accept-Encoding': 'gzip, deflate',
                         'Connection': 'keep-alive',
                         'Content-Length': '9',
                         'Content-Type': 'application/json',
                         'User-Agent': 'nti.webhooks...'},
             'method': 'POST',
             'url': 'https://example.com/some/path'},
 'response': {'Class': 'WebhookDeliveryAttemptResponse',
              'CreatedTime': ...,
              'Last Modified': ...,
              'MimeType': 'application/vnd.nextthought.webhooks.webhookdeliveryattemptresponse',
              'content': '',
              'elapsed': 'PT0...S',
              'headers': {'Content-Type': 'text/plain'},
              'reason': 'Not Found',
              'status_code': 404},
 'status': 'failed'}

API Reference¶

nti.webhooks.interfaces¶

Interface definitions for nti.webhooks.

interface nti.webhooks.interfaces.ICreatedTime[source]¶

Something that (immutably) tracks its created time.

createdTime¶

The timestamp at which this object was created.

Typically set automatically by the object.

Implementation:	`zope.schema.Real`
Read Only:	False
Required:	True
Default Value:	0.0
Allowed Type:	`numbers.Real`

interface nti.webhooks.interfaces.ILastModified[source]¶

Extends: nti.webhooks.interfaces.ICreatedTime

Something that tracks a modification timestamp.

lastModified¶

The timestamp at which this object or its contents was last modified.

Implementation:	`zope.schema.Real`
Read Only:	False
Required:	True
Default Value:	0.0
Allowed Type:	`numbers.Real`

interface nti.webhooks.interfaces.ILimitedApplicabilityPreconditionFailureWebhookSubscription[source]¶

Extends: nti.webhooks.interfaces.IWebhookSubscription

A webhook subscription that supports a limit on the number of times checking applicability can be allowed to fail.

When this number is exceeded, an event implementing IWebhookSubscriptionApplicabilityPreconditionFailureLimitReached is notified.

applicable_precondition_failure_limit¶: An integer giving the number of times applicability checks can fail before the event is generated.

interface nti.webhooks.interfaces.ILimitedAttemptWebhookSubscription[source]¶

Extends: nti.webhooks.interfaces.IWebhookSubscription

A webhook subscription that should limit the number of delivery attempts it stores.

attempt_limit¶: An integer giving approximately the number of delivery attempts this object will store. This is also used to deactivate the subscription when this many attempts in a row have failed.

interface nti.webhooks.interfaces.IPossibleWebhookPayload[source]¶

Marker interface applied to objects that may have webhook subscriptions defined for them.

The default configuration in subscribers.zcml loads event dispatchers only for event targets that implement this interface.

interface nti.webhooks.interfaces.IWebhookDeliveryAttempt[source]¶

Extends: nti.webhooks.interfaces._ITimes, zope.location.interfaces.IContained

The duration of the request/reply cycle is roughly captured by the difference in the createdTime attributes of the request and response. More precisely, the network time is captured by the elapsed attribute of the response.

__parent__¶

Implementation:	`zope.schema.Field`
Read Only:	False
Required:	True
Default Value:	None

status¶

The status of the delivery attempt.

The current status of the delivery attempt.

Attempts begin in the ‘pending’ state, and then transition to either the ‘successful’, or ‘failed’ state.

Implementation:	`nti.webhooks.interfaces._StatusField`
Read Only:	False
Required:	True
Default Value:	‘pending’

message¶

Additional explanatory text.

Implementation:	`nti.schema.field.ValidText`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

internal_info¶

Implementation:	`nti.schema.field.Object`
Read Only:	False
Required:	True
Default Value:	None
Must Provide:	`nti.webhooks.interfaces.IWebhookDeliveryAttemptInternalInfo`

request¶

Implementation:	`nti.schema.field.Object`
Read Only:	False
Required:	True
Default Value:	None
Must Provide:	`nti.webhooks.interfaces.IWebhookDeliveryAttemptRequest`

response¶

Implementation:	`nti.schema.field.Object`
Read Only:	False
Required:	True
Default Value:	None
Must Provide:	`nti.webhooks.interfaces.IWebhookDeliveryAttemptResponse`

succeeded()¶: Did the attempt succeed?

failed()¶: Did the attempt fail?

pending()¶: Is the attempt still pending?

resolved()¶: Has the attempt been resolved, one way or the other?

interface nti.webhooks.interfaces.IWebhookDeliveryAttemptFailedEvent[source]¶

Extends: nti.webhooks.interfaces.IWebhookDeliveryAttemptResolvedEvent

A delivery attempt failed.

The succeeded attribute will be false.

interface nti.webhooks.interfaces.IWebhookDeliveryAttemptInternalInfo[source]¶

Extends: nti.webhooks.interfaces._ITimes

Internal (debugging) information stored with a delivery attempt.

This data is never externalized and is only loosely specified.

It may change over time.

exception_history¶: A sequence (oldest to newest) of information about exceptions encountered processing the attempt.

originated¶: Information about where and how the request originated. This can be used to see if it might still be pending or if the instance has gone away.

interface nti.webhooks.interfaces.IWebhookDeliveryAttemptRequest[source]¶

Extends: nti.webhooks.interfaces._ITimes

The details about an HTTP request sent to a webhook.

url¶

The URL requested

This is denormalized from the containing delivery attempt and its containing subscription because the target URL may change over time.

Implementation:	`nti.schema.field.ValidURI`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

method¶

The HTTP method the request was sent with.

Implementation:	`nti.schema.field.ValidText`
Read Only:	False
Required:	True
Default Value:	‘POST’
Allowed Type:	`str`

body¶

The external data sent to the destination.

Implementation:	`nti.schema.field.ValidText`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

headers¶

The headers sent with the request.

Order is not kept. Security sensitive headers, such as those relating to authentication, are removed.

Implementation:	`zope.schema.Dict`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`dict`

Key Type

Implementation:	`nti.schema.field.ValidText`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

Value Type

Implementation:	`nti.schema.field.ValidText`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

interface nti.webhooks.interfaces.IWebhookDeliveryAttemptResolvedEvent[source]¶

Extends: zope.lifecycleevent.interfaces.IObjectModifiedEvent

A pending webhook delivery attempt has been completed.

This is an object modified event; the object is the attempt.

This is the root of a hierarchy; more specific events are in IWebhookDeliveryAttemptFailedEvent and IWebhookDeliveryAttemptSucceededEvent.

succeeded¶

Was the delivery attempt successful?

Implementation:	`zope.schema.Bool`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`bool`

interface nti.webhooks.interfaces.IWebhookDeliveryAttemptResponse[source]¶

Extends: nti.webhooks.interfaces._ITimes

The details about the HTTP response.

HTTP redirect history is lost; only the final response is saved.

status_code¶

The HTTP status code

For example, 200.

Implementation:	`nti.schema.field.Int`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`int`

reason¶

The HTTP reason.

For example, ‘OK’

Implementation:	`nti.schema.field.ValidText`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

headers¶

The headers received from the server.

Implementation:	`zope.schema.Dict`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`dict`

Key Type

Implementation:	`nti.schema.field.ValidText`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

Value Type

Implementation:	`nti.schema.field.ValidText`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

content¶

The decoded contents of the response, if any.

If the response contained a body, but it wasn’t decodable as text, XXX: What?

TODO: Place some limits on this?

Implementation:	`nti.schema.field.ValidText`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

elapsed¶

The amount of time it took to send and receive.

This should be the closest measurement possible of the time taken between sending the first byte of the request, and receiving a usable response.

Implementation:	`zope.schema.Timedelta`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`datetime.timedelta`

interface nti.webhooks.interfaces.IWebhookDeliveryAttemptSucceededEvent[source]¶

Extends: nti.webhooks.interfaces.IWebhookDeliveryAttemptResolvedEvent

A delivery attempt succeeded.

The succeeded attribute will be true.

interface nti.webhooks.interfaces.IWebhookDeliveryManager[source]¶

Handles the delivery of messages.

This is a global utility registered by the ZCML of this package.

It operates in fire-and-forget mode, in a completely opaque fashion. However, this is a two-step process to better work with persistent objects and transactions. In the first step, a IWebhookDeliveryManagerShipmentInfo is created with createShipmentInfo(), packaging up all the information needed to later begin the delivery using acceptForDelivery().

(And yes, the terminology is based on the United States Postal Service.)

createShipmentInfo(subscriptions_and_attempts)¶

Given an (distinct) iterable of (subscription, attempt) pairs, extract the information needed to later send that data as well as record its status in the subscription, independently of any currently running transaction or request.

Each attempt must be pending and must not be contained in any other shipment info.

For persistent subscriptions and attempts, all necessary information to complete acceptForDelivery() must be captured at this time. The connection that created the subscription and attempts must still be open, and the transaction still running.

Returns:	A new `IWebhookDeliveryManagerShipmentInfo` object. If the iterable is empty, this may return None or a suitable object that causes `acceptForDelivery()` to behave appropriately.

acceptForDelivery(shipment_info)¶

Given a IWebhookDeliveryManagerShipmentInfo previously created by this object but not yet accepted for delivery, schedule the delivery and begin making it happen.

This is generally an asynchronous call and SHOULD NOT raise exceptions; the caller is likely unable to deal with them.

As delivery completes, the status of each attempt contained in the shipment info should be updated.

No return value.

interface nti.webhooks.interfaces.IWebhookDeliveryManagerShipmentInfo[source]¶: A largely-opaque interface representing values returned from, and passed to, a particular IWebhookDeliveryManager.

interface nti.webhooks.interfaces.IWebhookDestinationValidator[source]¶

Validates destinations.

This is the place where we make sure that the destination is valid, before attempting to deliver to it, according to policy. This may include such things as:

Check that the protocol is HTTPs.
Verify that the domain is reachable, or at least resolvable.
Ensure query parameters are innocuous

Targets are validated before attempting to send data to them.

This is registered as a single global utility. The utility is encouraged to cache valid/invalid results for a period of time, especially with domain resolvability.

validateTarget(target_url)¶

Check that the URL is valid. If it is, return silently.

If it is not, raise some sort of exception such as a socket.error for unresolvable domains.

interface nti.webhooks.interfaces.IWebhookDialect[source]¶

Provides control over what data is sent on the wire.

externalizeData(data, event)¶

Produce the byte-string that is the externalized version of data needed to send to webhooks using this dialect.

This is called while the transaction that triggered the event is still open and not yet committed.

The default method will externalize the data using an nti.externalization externalizer named “webhook-delivery”.

prepareRequest(http_session, subscription, attempt)¶

Produce the prepared request to send.

Parameters:

http_session (requests.Session) – The session being used to send requests. The implementation should generally create a requests.Request object, and then prepare it with requests.Session.prepare_request() to combine the two.
subscription (IWebhookSubscription) – The subscription that is being delivered.
attempt (IWebhookDeliveryAttempt) – The attempt being sent. It will already have its payload_data, which should be given as the data argument to the request.

Return type:

requests.PreparedRequest

Caution

It may not be possible to access attributes of persistent objects

interface nti.webhooks.interfaces.IWebhookPayload[source]¶: Adapter interface to convert an object that is a target of an event (possibly a IPossibleWebhookPayload) into the object that should actually be used as the payload.

interface nti.webhooks.interfaces.IWebhookPrincipal[source]¶

A minimal version of zope.security.interfaces.IPrincipal.

This is used by this package when we need to convert an arbitrary object into something that can match up with a owner_id, as used by IWebhookSubscription. It is useful if your own objects don’t adapt to or implement IPrincipal.

See also

nti.webhooks.subscribers.remove_subscriptions_for_principal

id¶

Id

The unique identification of the principal.

Implementation:	`nti.schema.field.ValidTextLine`
Read Only:	True
Required:	True
Default Value:	None
Allowed Type:	`str`

interface nti.webhooks.interfaces.IWebhookResourceDiscriminator[source]¶

An adapter that can figure out a better for for a resource than simply what it provides.

__call__()¶: Return the value to use for for.

interface nti.webhooks.interfaces.IWebhookSubscription[source]¶

Extends: nti.webhooks.interfaces._ITimes, zope.container.interfaces.IContainerNamesContainer

An individual subscription.

__parent__¶

Implementation:	`zope.schema.Field`
Read Only:	False
Required:	True
Default Value:	None

for_¶

The type of object to attempt delivery for.

When object events of type when are fired for instances providing this interface, webhook delivery to target might be attempted.

The default is objects that implement IWebhookPayload.

This is interpreted as for zope.component.registerAdapter() and may name an interface or a type.

Implementation:	`zope.schema.Field`
Read Only:	False
Required:	True
Default Value:	<InterfaceClass nti.webhooks.interfaces.IWebhookPayload>

when¶

The type of event that should result in attempted deliveries.

A type of IObjectEvent, usually one defined in zope.lifecycleevent.interfaces such as IObjectCreatedEvent. The object field of this event must provide the for_ interface; it’s the data from the object field of this event that will be sent to the webhook.

If not specified, all object events involving the for_ interface will be sent.

This must be an interface.

Implementation:	`nti.schema.field.Object`
Read Only:	False
Required:	True
Default Value:	<InterfaceClass zope.interface.interfaces.IObjectEvent>
Must Provide:	`zope.interface.interfaces.IInterface`

to¶

The complete destination URL to which the data should be sent

This is an arbitrary HTTPS URL. Only HTTPS is supported for delivery of webhooks.

Implementation:	`nti.webhooks._schema.HTTPSURL`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

owner_id¶

The ID of the IPrincipal that owns this subscription.

This will be validated at runtime when an event arrives. If the current zope.security.interfaces.IAuthentication utility cannot find a principal with the given ID, the delivery will be failed.

Leave unset to disable security checks.

This cannot be changed after creation.

Implementation:	`nti.webhooks._schema.PermissivePrincipalId`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

permission_id¶

The ID of the permission to check

If given, and an owner is also specified, then only data that has this permission for the owner will result in an attempted delivery. If not given, but an owner is given, this will default to the standard view permission ID, zope.View.

If the permission ID cannot be found at runtime, the delivery will fail.

Implementation:	`nti.schema.field.ValidChoice`
Read Only:	False
Required:	False
Default Value:	None

dialect_id¶

The ID of the IWebhookDialect to use

Dialects are named utilities. They control the authentication, headers, and HTTP method.

Implementation:	`nti.schema.field.ValidChoice`
Read Only:	False
Required:	False
Default Value:	None

dialect¶: The resolved dialect to use for this subscription.

active¶

Is this webhook active? (Registered to process events.)

Determined by the subscription manager that owns this subscription.

Implementation:	`zope.schema.Bool`
Read Only:	True
Required:	False
Default Value:	True
Allowed Type:	`bool`

status_message¶

Explanatory text about the state of this subscription.

Implementation:	`nti.schema.field.ValidText`
Read Only:	False
Required:	True
Default Value:	‘Active’
Allowed Type:	`str`

__setitem__(key, value)¶

Add the given object to the container under the given name.

Raises a TypeError if the key is not a unicode or ascii string.

Raises a ValueError if the key is empty, or if the key contains a character which is not allowed in an object name.

Raises a KeyError if the key violates a uniqueness constraint.

The container might choose to add a different object than the one passed to this method.

If the object doesn’t implement IContained, then one of two things must be done:

If the object implements ILocation, then the IContained interface must be declared for the object.
Otherwise, a ContainedProxy is created for the object and stored.

The object’s __parent__ and __name__ attributes are set to the container and the given name.

If the old parent was None, then an IObjectAddedEvent is generated, otherwise, an IObjectMovedEvent is generated. An IContainerModifiedEvent is generated for the container.

If the object replaces another object, then the old object is deleted before the new object is added, unless the container vetos the replacement by raising an exception.

If the object’s __parent__ and __name__ were already set to the container and the name, then no events are generated and no hooks. This allows advanced clients to take over event generation.

isApplicable(data)¶

Determine if this subscription applies to the given data object.

This does not take into account whether this subscription is active or not, but does take into account the permission and principal declared for the subscription as well as the type/interface.

This is a query method that does not mutate this object.

createDeliveryAttempt(payload_data)¶

Create a new IWebhookDeliveryAttempt for this subscription.

The delivery attempt is in the pending status, and is stored as a child of this subscription; its __parent__ is set to this subscription.

Subscriptions may be limited in the amount of attempts they will store; this method may cause that size to temporarily be exceeded

interface nti.webhooks.interfaces.IWebhookSubscriptionManager[source]¶

Extends: nti.webhooks.interfaces._ITimes, nti.webhooks.interfaces.IWebhookSubscriptionRegistry, zope.container.interfaces.IContainerNamesContainer

A utility that manages subscriptions.

Also a registry for which subscriptions fire on what events.

__setitem__(key, value)¶

Add the given object to the container under the given name.

Raises a TypeError if the key is not a unicode or ascii string.

Raises a ValueError if the key is empty, or if the key contains a character which is not allowed in an object name.

Raises a KeyError if the key violates a uniqueness constraint.

The container might choose to add a different object than the one passed to this method.

If the object doesn’t implement IContained, then one of two things must be done:

If the object implements ILocation, then the IContained interface must be declared for the object.
Otherwise, a ContainedProxy is created for the object and stored.

The object’s __parent__ and __name__ attributes are set to the container and the given name.

If the old parent was None, then an IObjectAddedEvent is generated, otherwise, an IObjectMovedEvent is generated. An IContainerModifiedEvent is generated for the container.

If the object replaces another object, then the old object is deleted before the new object is added, unless the container vetos the replacement by raising an exception.

If the object’s __parent__ and __name__ were already set to the container and the name, then no events are generated and no hooks. This allows advanced clients to take over event generation.

createSubscription(to=None, for_=None, when=None, owner_id=None, permission_id=None, dialect=None)¶

Create and store a new IWebhookSubscription in this manager.

The new subscription is returned. It is a child of this object.

All arguments are by keyword, and have the same meaning as the attributes documented for IWebhookSubscription.

Newly created subscriptions are always active.

deactivateSubscription(subscription)¶: Given a subscription managed by this object, deactivate it.

activateSubscription(subscription)¶: Given a subscription managed by this object, activate it.

deleteSubscriptionsForPrincipal(principal_id)¶

Remove all subscriptions in this manager owned by principal_id.

(This is the same as the owner_id parameter to createSubscription().)

interface nti.webhooks.interfaces.IWebhookSubscriptionManagers[source]¶

Used as an adapter to provide an iterable of potentially interesting or related subscription managers.

See also

nti.webhooks.subscribers.remove_subscriptions_for_principal

__iter__()¶: Provide an iterator over IWebhookSubscriptionManagers.

interface nti.webhooks.interfaces.IWebhookSubscriptionSecuritySetter[source]¶

An adapter for the subscription that sets initial security declarations for a subscription.

The subscription is also passed to the call method to allow for simple functions to be used as the adapter.

In the future, the call method might also accept an event argument, and the request might be passed as a second argument to the constructor.

__call__(subscription)¶: Set the security declarations for the subscription.

nti.webhooks.api¶

API functions.

nti.webhooks.api.subscribe_in_site_manager(site_manager, subscription_kwargs, utility_name='WebhookSubscriptionManager')[source]¶

Produce and return a persistent IWebhookSubscription in the given site manager.

The subscription_kwargs are as for nti.webhooks.interfaces.IWebhookSubscriptionManager.createSubscription(). No defaults are applied here.

The utility_name can be used to namespace subscriptions. It must never be empty.

nti.webhooks.api.subscribe_to_resource(resource, to, for_=None, when=<InterfaceClass zope.interface.interfaces.IObjectEvent>, dialect_id=None, owner_id=None, permission_id=None)[source]¶

Produce and return a persistent IWebhookSubscription based on the resource.

Only the resource and to arguments are mandatory. The other arguments are optional, and are the same as the attributes in that interface.

Parameters:

resource –

The resource to subscribe to. Passing a resource does two things. First, the resources is used to find the closest enclosing ISite that is persistent. A IWebhookSubscriptionManager utility will be installed in this site if one does not already exist, and the subscription will be created there.

Second, if for_ is not given, then the interfaces provided by the resource will be used for for_. This doesn’t actually subscribe just to events on that exact object, but to events for objects with the same set of interfaces.

nti.webhooks.dialect¶

Implementations of dialects.

class nti.webhooks.dialect.DefaultWebhookDialect[source]¶

Bases: object

Default implementation of a nti.webhooks.interfaces.IWebhookDialect.

This class is intended to be subclassed; other dialect implementations should extend this class. This permits freedom in adding additional methods to the interface.

externalizeData(data, event)[source]¶: See nti.webhooks.interfaces.IWebhookDialect.externalizeData()

prepareRequest(http_session, subscription, attempt)[source]¶: See nti.webhooks.interfaces.IWebhookDialect.prepareRequest()

produce_payload(data, event) → IWebhookPayload[source]¶

Non-interface method. Given data delivered through an event, try to find a IWebhookPayload for it. From highest to lowest priority, this means:

A multi-adapter from the object and the event named externalizer_name.
The unnamed multi-adapter.
A single adapter from the object named externalizer_name
The unnamed single adapter.

The data is used as the context for the lookup in all cases. XXX: Or maybe we should use the subscription as the context?

Note that if there exists an adapter registration that returns None, we continue with lower-priority adapters.

content_type = 'application/json'¶: The MIME type of the body produced by externalizeData(). If you change the externalizer_format, you need to change this value.

externalizer_format = 'json'¶: Which representation to use. Passed to nti.externaliaztion.to_external_representation()

externalizer_name = 'webhook-delivery'¶: The name of the externalizer used to produce the external form. This is also the highest-priority name of the adapter used.

externalizer_policy_name = 'webhook-delivery'¶: The name of the externalization policy utility used to produce the external form. This defaults to one that uses ISO8601 format for Unix timestamps.

http_method = 'POST'¶: The HTTP method (verb) to use.

user_agent = 'nti.webhooks 0.0.7.dev0'¶: The HTTP “User-Agent” header.

nti.webhooks.delivery_manager¶

Default implementation of the delivery manager.

interface nti.webhooks.delivery_manager.IExecutorService[source]¶

Internal interface for testing. See nti.webhooks.testing.SequentialExecutorService for the alternate implementation.

submit(func)¶: Submit a job to run. Execution may or may not commence in the background. Tracks tasks that have been submitted and not yet finished.

waitForPendingExecutions(timeout=None)¶: Wait for all tasks that have been submitted but not yet finished to finish. submit, wait for them all to finish. If any one of them raises an exception, this method should raise an exception.

shutdown()¶: Stop accepting new tasks.

class nti.webhooks.delivery_manager.DefaultDeliveryManager(name)[source]¶: Bases: zope.container.contained.Contained

nti.webhooks.subscriptions¶

Subscription implementations.

interface nti.webhooks.subscriptions.IApplicableSubscriptionFactory[source]¶

A private contract between the Subscription and its SubscriptionManager.

This is only called on subscriptions that are already determined to be active; if the subscription is also applicable, then it should be returned. Otherwise, it should return None.

This is called when we intend to attempt delivery, so it’s a good time to take cleanup action if the subscription isn’t applicable for reasons that aren’t directly related to the data and the event, for example, if the principal cannot be found.

__call__(data, event)¶: See class documentation.

class nti.webhooks.subscriptions.AbstractSubscription(**kwargs)[source]¶

Bases: nti.schema.schema.SchemaConfigured

Subclasses need to extend a Container implementation.

clear()[source]¶: Testing only. Removes all delivery attempts.

pop()[source]¶: Testing only. Removes and returns a random value.

class nti.webhooks.subscriptions.GlobalSubscriptionComponents(name='', bases=())[source]¶

Bases: zope.component.globalregistry.BaseGlobalComponents

Exists to be pickled by name.

class nti.webhooks.subscriptions.GlobalWebhookSubscriptionManager(name)[source]¶: Bases: nti.webhooks.subscriptions.AbstractWebhookSubscriptionManager, nti.webhooks.subscriptions._CheckObjectOnSetSampleContainer, nti.webhooks._util.DCTimesMixin

class nti.webhooks.subscriptions.PersistentSubscription(**kwargs)[source]¶

Bases: nti.webhooks.subscriptions._CheckObjectOnSetBTreeContainer, nti.webhooks.subscriptions.AbstractSubscription, nti.webhooks._util.PersistentDCTimesMixin

Persistent implementation of IWebhookSubscription

class nti.webhooks.subscriptions.PersistentWebhookSubscriptionManager[source]¶: Bases: nti.webhooks.subscriptions.AbstractWebhookSubscriptionManager, nti.webhooks._util.PersistentDCTimesMixin, nti.webhooks.subscriptions._CheckObjectOnSetBTreeContainer

class nti.webhooks.subscriptions.Subscription(**kwargs)[source]¶: Bases: nti.webhooks.subscriptions._CheckObjectOnSetSampleContainer, nti.webhooks.subscriptions.AbstractSubscription, nti.webhooks._util.DCTimesMixin

nti.webhooks.subscribers¶

Event subscribers.

class nti.webhooks.subscribers.ExhaustiveWebhookSubscriptionManagers(context)[source]¶

Bases: object

Finds all subscription managers that are located in the same root as the context.

This is done using an exhaustive, expensive process of adapting the root to zope.container.interfaces.ISublocations and inspecting each of them for subscription managers.

This is not registered by default.

nti.webhooks.subscribers.dispatch_webhook_event(data, event)[source]¶

A subscriber installed to dispatch events to webhook subscriptions.

This is usually registered in the global registry by loading subscribers.zcml or subscribers_promiscuous.zcml, but the event and data for which it is registered may be easily customized. See Configuration for more information.

This function:

Queries for all active subscriptions in the IWebhookSubscriptionManager instances in the current site hierarchy;

And queries for all active subscriptions in the IWebhookSubscriptionManager instances in the context of the data, which may be separate.

Determines if any of those actually apply to the data, and if so, joins the transaction to prepare for sending them.

Important

Checking whether a subscription is applicable depends on the security policy in use. Most security policies inspect the object’s lineage or location (walking up the __parent__ tree) so it’s important to use this subscriber only for events where that part of the object is intact. For example, it does not usually apply for ObjectCreatedEvent, but does for ObjectAddedEvent. See configuration for more.

Caution

This function assumes the global, thread-local transaction manager. If any objects belong to ZODB connections that are using a different transaction manager, this won’t work.

nti.webhooks.subscribers.remove_subscriptions_for_principal(principal, event)[source]¶

Subscriber to find and remove all subscriptions for the principal when it is removed.

This is an adapter for (IPrincipal, IObjectRemovedEvent) by default, but that may not be the correct event in every system. Register it for the appropriate events in your system.

Parameters:

principal –
The principal being removed. It should still be located (having a proper __parent__) when this subscriber is invoked; this is the default for zope.container objects that use zope.container.contained.uncontained() in their __delitem__ method.

This can be any type of object. It is first adapted to nti.webhooks.interfaces.IWebhookPrincipal; if that fails, it is adapted to IPrincipal, and if that fails, it is used as-is. The final object must have the id attribute.
event – This is not used by this subscriber.

This subscriber removes all subscriptions owned by the principal found in subscription managers:

in the current site; and
in sites up the lineage of the original principal and adapted object (if different).

If the principal may have subscriptions in more places, provide an implementation of nti.webhooks.interfaces.IWebhookSubscriptionManagers for the original principal object. One (exhaustive) implementation is provided (but not registered) in ExhaustiveWebhookSubscriptionManagers.

nti.webhooks.zcml¶

Support for configuring webhook delivery using ZCML.

interface nti.webhooks.zcml.IDialectDirective[source]¶

Create a new dialect subclass of DefaultWebhookDialect and register it as a utility named name.

name¶

Name

Name of the dialect registration. Limited to ASCII characters.

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

externalizer_name¶

The name of the externalization adapters to use

Remember, if adapters by this name do not exist, the default will be used.

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

externalizer_policy_name¶

The name of the externalizer policy component to use.

Important

An empty string selects the nti.externalization default policy, which uses Unix timestamps. To use the default policy of nti.webhooks, omit this argument.

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

http_method¶

The HTTP method to use.

This should be a valid method name, but that’s not enforced

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

user_agent¶

The User-Agent header string to use.

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

interface nti.webhooks.zcml.IStaticPersistentSubscriptionDirective[source]¶

Extends: nti.webhooks.zcml.IStaticSubscriptionDirective

Define a local, static, persistent subscription.

Local persistent subscriptions live in the ZODB database, beneath some zope.site.interfaces.ILocalSiteManager.

They are identified by a traversable path beginning from the root of the database; note that this may not be the exact same as a path exposed in the application because this path will need to include the name of the root application object, while application paths typically do not.

This package uses zope.generations to keep track of registered subscriptions and synchronize the database with what is in executed ZCML. Thus it is very important not to remove ZCML directives, or only execute part of the ZCML configuration unless you intend for the subscriptions not found in ZCML to be removed.

All the options are the same as for IStaticSubscriptionDirective, with the addition of the required site_path.

site_path¶

The path to traverse to the site

A persistent subscription manager will be installed in this site.

Implementation:	`nti.webhooks.zcml.Path`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

interface nti.webhooks.zcml.IStaticSubscriptionDirective[source]¶

Define a global, static, transient subscription.

Static subscriptions are not persistent and live only in the memory of individual processes. Thus, failed deliveries cannot be re-attempted after process shutdown. And of course the delivery history is also transient and local to a process.

for_¶

The type of object to attempt delivery for.

When object events of type when are fired for instances providing this interface, webhook delivery to target might be attempted.

The default is objects that implement IWebhookPayload.

This is interpreted as for zope.component.registerAdapter() and may name an interface or a type.

Implementation:	`zope.configuration.fields.GlobalObject`
Read Only:	False
Required:	False
Default Value:	<InterfaceClass nti.webhooks.interfaces.IWebhookPayload>

when¶

The type of event that should result in attempted deliveries.

A type of IObjectEvent, usually one defined in zope.lifecycleevent.interfaces such as IObjectCreatedEvent. The object field of this event must provide the for_ interface; it’s the data from the object field of this event that will be sent to the webhook.

If not specified, all object events involving the for_ interface will be sent.

This must be an interface.

Implementation:	`nti.webhooks._schema.ObjectEventInterface`
Read Only:	False
Required:	False
Default Value:	<InterfaceClass zope.interface.interfaces.IObjectEvent>

Value Type

Implementation:	`nti.webhooks._schema.ObjectEventField`
Read Only:	False
Required:	True
Default Value:	None

to¶

The complete destination URL to which the data should be sent

This is an arbitrary HTTPS URL. Only HTTPS is supported for delivery of webhooks.

Implementation:	`nti.webhooks._schema.HTTPSURL`
Read Only:	False
Required:	True
Default Value:	None
Allowed Type:	`str`

dialect¶

The ID of the IWebhookDialect to use

Dialects are named utilities. They control the authentication, headers, and HTTP method.

Implementation:	`zope.schema.TextLine`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

owner¶

The ID of the IPrincipal that owns this subscription.

This will be validated at runtime when an event arrives. If the current zope.security.interfaces.IAuthentication utility cannot find a principal with the given ID, the delivery will be failed.

Leave unset to disable security checks.

This cannot be changed after creation.

Implementation:	`nti.webhooks._schema.PermissivePrincipalId`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

permission¶

The permission to check

If given, and an owner is also specified, then only data that has this permission for the owner will result in an attempted delivery. If not given, but an owner is given, this will default to the standard view permission ID, zope.View.

Implementation:	`zope.security.zcml.Permission`
Read Only:	False
Required:	False
Default Value:	None
Allowed Type:	`str`

class nti.webhooks.zcml.Path(*args, **kw)[source]¶

Bases: zope.schema._bootstrapfields.Text

Accepts a single absolute traversable path.

Unlike zope.configuration.fields.Path, this version requires that the path be absolute and uses URL separators.

fromUnicode(value)[source]¶

>>> from zope.schema import Text
>>> t = Text(constraint=lambda v: 'x' in v)
>>> t.fromUnicode(b"foo x spam") # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
zope.schema._bootstrapinterfaces.WrongType:
    ('foo x spam', <type 'unicode'>, '')
>>> result = t.fromUnicode(u"foo x spam")
>>> isinstance(result, bytes)
False
>>> str(result)
'foo x spam'
>>> t.fromUnicode(u"foo spam") # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
zope.schema._bootstrapinterfaces.ConstraintNotSatisfied:
    (u'foo spam', '')

nti.webhooks.testing¶

Changes¶

0.0.7 (unreleased)¶

Nothing changed yet.

0.0.6 (2021-09-07)¶

Make subscriptions, delivery attempts, delivery attempt requests, and delivery attempt responses have a mimeType value when externalized.

0.0.5 (2020-12-04)¶

Add support for Python 3.9.
Principal IDs are no longer required to be URIs or dotted names. See issue 21.

0.0.4 (2020-09-16)¶

Use a custom ITraverser when finding sites to install persistent ZCML subscriptions in. This traverser fires IBeforeTraverseEvent notifications, letting subscribers to that (such as nti.site.subscribers.threadSiteSubscriber) take action (such as making sites current when they’re about to be traversed). This can help when the site path contains namespaces.

0.0.3 (2020-08-24)¶

Move permission definition to a separate file, permissions.zcml, that is included by default. Use the ZCML <exclude> directive before including this package’s configuration if you were experiencing configuration conflicts.

0.0.2 (2020-08-06)¶

Add a subscriber and methods to remove subscriptions when principals are deleted. See PR 17.

0.0.1 (2020-08-05)¶

Initial PyPI release.

TODO ¶

Todo

Write events document. Add specific events for subscription in/activated.

Todo

The concept of an active, in-process, retry policy.

Todo

An API to retry a failed request.

Dynamic-subscriptions only ¶

Thoughts on HTTP API ¶

Todo

Generic end-point with context IPossibleWebhookPayload; the last part of the path (or a query param?) would be a shortcut name for the event.

Todo

Getting the attempts for a subscription should be easy. Have an endpoint for the subscription, just externalize.

nti.webhooks¶

Webhooks¶

This Package¶

Data Model (Subscription Combinations)¶

Documentation¶

Scope¶

Out Of Scope¶

In Scope/Features¶

Glossary¶

Configuration¶

Recommended: subscribers.zcml¶

Development and Testing: subscribers_promiscuous.zcml¶

Configured Global, Transient Webhook Subscriptions¶

ZCML Directive Arguments¶

Specifying Which Objects¶

Active Subscriptions¶

Delivery Attempts¶

Unsuccessful Delivery Attempts¶

Don’t Fail The Transaction¶

Inactive Subscriptions¶

Configured Local, Persistent Webhook Subscriptions¶

Adding A Subscription¶

Delivering To The Subscription¶

Mutating Subscriptions¶

Delivery Security Checks¶

Owners¶

Permissions¶

Example¶

Subscription Is Not Applicable By Default¶

Subscription Applicable Once Principals are Defined¶

Existing Interactions¶

Automatic Deactivation On Failure¶

Delivery History¶

Types of Delivery Attempts¶

Successful Delivery Attempts¶

Attempt Details¶

Failed Delivery Attempts¶

Pending Delivery Attempts¶

Limits On History¶

Automatic Deactivation on Failures¶

HTTP Failures¶

Local Failures¶

Validation Failures¶

Subscription Security¶

Set Up¶

Access To The Subscription¶

Access to Delivery Attempts¶

Security Proxies¶

Customizing The Permission Grants¶

Customizing HTTP Requests¶

Customizing The Body¶

Single Adapters¶

Multi-adapters¶

Cleanup¶

Webhook Dialects¶

Setting the Body¶

Setting Headers¶

Defining A Dialect Using ZCML¶

Dynamic Webhook Subscriptions¶

Goals¶

Customizing For¶

Example¶

Setup¶

High-level API¶

What Just Happened¶

A Subscription Manager Was Created¶

The for Was Inferred¶

The when Was Guessed¶

The Subscription is Active¶

Relation To Static Subscriptions¶

Delivery to Static and Dynamic Subscriptions¶

Removing Subscriptions¶

Principal Removal¶

Registering The Handler¶

Subscription Managers Outside The Site and Lineage¶

Events¶

Externalization¶

Externalizing a Subscription¶

API Reference¶

nti.webhooks.interfaces¶

Webhooks ¶

This Package ¶

Documentation ¶

Recommended: `subscribers.zcml`¶

Development and Testing: `subscribers_promiscuous.zcml`¶

The `for` Was Inferred¶

The `when` Was Guessed¶

TODO ¶

Dynamic-subscriptions only ¶

Thoughts on HTTP API ¶