From 827e3cd7f010815ebd02ee97c94697e2e7a6d456 Mon Sep 17 00:00:00 2001
From: Philip Withnall <philip.withnall@collabora.co.uk>
Date: Fri, 6 Feb 2015 09:38:07 +0000
Subject: [PATCH 2/3] fixup! doc: Add a guide to designing D-Bus APIs

---
 doc/dbus-api-design.duck | 149 ++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 115 insertions(+), 34 deletions(-)

diff --git a/doc/dbus-api-design.duck b/doc/dbus-api-design.duck
index b3224ac..90e9482 100644
--- a/doc/dbus-api-design.duck
+++ b/doc/dbus-api-design.duck
@@ -23,9 +23,11 @@
   libraries like
   $link[>>https://developer.gnome.org/gio/stable/gdbus-convenience.html](GDBus),
   but detailed implementation instructions are left to the libraries’
-  documentation. Note that you should $em(not) use libdbus-1 or libdbus-glib to
-  implement D-Bus services, as they are awkward to use correctly, deprecated
-  and unmaintained.
+  documentation. Note that you should $em(not) use dbus-glib to implement D-Bus
+  services as it is deprecated and unmaintained. Most services should also avoid
+  libdbus (dbus-1), which is a low-level library and is awkward to use
+  correctly: it is designed to be used via a language binding such as
+  $link[>>http://qt-project.org/doc/qt-4.8/qtdbus.html](QtDBus).
 
   For documentation on D-Bus itself, see the
   $link[>>http://dbus.freedesktop.org/doc/dbus-specification.html](D-Bus
@@ -57,8 +59,9 @@ readable way. The format is described in the
 $link[>>http://dbus.freedesktop.org/doc/dbus-specification.html#introspection-format](D-Bus
 specification), and is supported by tools such as $cmd(gdbus-codegen).
 
-Interface files should be installed to $code($var($$(datadir$))/dbus-1/interfaces)
-so that other services can load them. There should be one file installed
+Interface files for public API should be installed to
+$code($var($$(datadir$))/dbus-1/interfaces) so that other services can load
+them. Private APIs should not be installed There should be one file installed
 per D-Bus interface, named after the interface, containing a single top-level
 $code(<node>) element with a single $code(<interface>) beneath it. For example,
 interface $code(org.example.MyService1.Manager) would be described by file
@@ -118,8 +121,11 @@ than inserting a number everywhere, which takes more effort.
 New API can be added to a D-Bus interface without incrementing the version
 number, as such additions are still backwards-compatible. However, clients
 should gracefully handle the $code(org.freedesktop.DBus.Error.UnknownMethod)
-error reply from all D-Bus method calls so that they can run against older
-versions of the service which don’t implement new methods.
+error reply from all D-Bus method calls if they want to run against older
+versions of the service which don’t implement new methods. (This also prevents
+use of generated bindings; any method which a client wants to gracefully fall
+back from should be called using a generic D-Bus method invocation rather than
+a specific generated binding.)
 
 When API is broken, changed or removed, the service’s version number must be
 bumped; for example, from $code(org.example.MyService1)
@@ -213,12 +219,22 @@ to expose as much structure in types as possible. In particular, sending
 structured strings over D-Bus should be avoided, as they need to be built and
 parsed; both are complex operations which are prone to bugs.
 
-A common, useful approach is to use enumerated values transmitted as unsigned
-integers over the bus, with the interface documentation describing what each
-value means, and how unrecognized values should be handled. This allows for
-extra values to be added in future without breaking API. Integers are better
-than strings in this case as they do not require parsing or string matching when
-received, and are more compact.
+For APIs being used in constrained situations, enumerated values should be
+transmitted as unsigned integers. For APIs which need to be extended by third
+parties or which are used in more loosely coupled systems, enumerated values
+should be strings in some defined format.
+
+Transmitting values as integers means string parsing and matching can be
+avoided, the messages are more compact, and typos can be more easily avoided by
+developers (if, for example, C enums are used in the implementation).
+
+Transmissing values as strings means additional values can be defined by third
+parties without fear of conflicting over integer values.
+
+In both cases, the interface documentation should describe the meaning of each
+value. It should state whether the type can be extended in future and, if so,
+how the service and client should handle unrecognized values — typically by
+considering them equal to a ‘unknown’ or ‘failure’ value.
 
 [example]
   For example, instead of:
@@ -284,6 +300,11 @@ be running in different locales, and hence interpret any human readable strings
 differently, or present them to the user in the wrong language. Transmit an
 enumerated value and convert it to a human readable string in the client.
 
+In situations where a service has received a human readable string from
+somewhere else, it should pass it on unmodified to the client, ideally with its
+locale alongside. Passing human readable information to a client is better than
+passing nothing.
+
 [example]
   For example, instead of:
   [code style="invalid" mime="application/xml"]
@@ -322,7 +343,8 @@ than signed values.
 
 Structures can be used almost anywhere in a D-Bus type, and arrays of structures
 are particularly useful. Structures should be used wherever data fields are
-related.
+related. Note, however, that structures are not extensible in future, so always
+consider $link[>#extensibility].
 
 [example]
   For example, instead of several identically-indexed arrays containing
@@ -367,6 +389,31 @@ is no need to null-terminate them or encode their length separately.
   FIXME: Mention maybe types and the extended kdbus/GVariant type system once
   that’s stable and round-trip-ability is no longer a concern.
 
+=== Extensibility
+    [id="extensibility"]
+
+Some D-Bus APIs have very well-defined use cases, and will never need extension.
+Others are used in more loosely coupled systems which may change over time, and
+hence should be designed to be extensible from the beginning without the need
+to break API in future. This is a trade off between having a more complex API,
+and being able to easily extend it in future.
+
+The key tool for extensibility in D-Bus is $code(a{sv}), the dictionary mapping
+strings to variants. If well-defined namespaced strings are used as the
+dictionary keys, arbitrary D-Bus peers can add whatever information they need
+into the dictionary. Any other peer which understands it can query and retrieve
+the information; other peers will ignore it.
+
+The canonical example of an extensible API using $code(a{sv}) is
+$link[>>http://telepathy.freedesktop.org/spec/](Telepathy). It uses $code(a{sv})
+values as the final element in structures to allow them to be extended in
+future.
+
+A secondary tool is the use of flag fields in method calls. The set of accepted
+flags is entirely under the control of the interface designer and, as with
+enumerated types, can be extended in future without breaking API. Adding more
+flags allows the functionality of the method call to be tweaked.
+
 === Using Signals, Properties and Errors
     [id="using-the-features"]
 
@@ -383,6 +430,12 @@ exactly once to each method call. Replying at the end of a long-running
 operation is correct — the client will patiently wait until the operation has
 finished and the reply is received.
 
+Note that D-Bus client bindings may implement synthetic timeouts of several
+tens of seconds, unless explicitly disabled for a call. For very long-running
+operations, you should disable the client bindings’ timeout and make it clear
+in the client’s UI that the application has not frozen and is simply running a
+long operation.
+
 An anti-pattern to avoid in this situation is to start a long-running operation
 when a method call is received, then to never reply to the method call and
 instead notify the client of completion of the operation via a signal. This
@@ -398,6 +451,10 @@ parameters, and having to return dummy values in the other parameters. Using
 D-Bus error replies also means such failures can be highlighted in debugging
 tools, simplifying debugging.
 
+Clients should handle all possible standard and documented D-Bus errors for each
+method call. IPC inherently has more potential failures than normal C function
+calls, and clients should be prepared to handle all of them gracefully.
+
 === Using Standard Interfaces
     [id="standard-interfaces"]
 
@@ -447,12 +504,18 @@ $code(Get$var(PropertyName)) and $code(Set$var(PropertyName)) methods.
 The specification also defines the
 $link[>>http://dbus.freedesktop.org/doc/dbus-specification.html#standard-interfaces-objectmanager]($code(org.freedesktop.DBus.ObjectManager))
 interface, which should be used whenever a service needs to expose a variable
-number of objects of the same class in a flat or tree-like structure. For
+number of objects of the same class in a flat or tree-like structure, and
+clients are expected to be interested in most or all of the objects. For
 example, this could be used by an address book service which exposes multiple
 address books, each as a separate object. The $code(GetManagedObjects) method
 allows the full object tree to be queried, returning all the objects’ properties
 too, eliminating the need for further IPC round trips to query the properties.
 
+If clients are not expected to be interested in most of the exposed objects,
+$code(ObjectManager) should $em(not) be used, as it will send all of the objects
+to each client anyway, wasting bus bandwidth. A file manager, therefore, should
+not expose the entire file system hierarchy using $code(ObjectManager).
+
 [example]
   For example, consider an object implementing an interface
   $code(org.example.MyService1.AddressBookManager) with methods:
@@ -736,23 +799,41 @@ $link[>#security-policies](security policy) as well.
 == Security Policies
    [id="security-policies"]
 
+At a high level, the D-Bus security model is:
+[list]
+* There is a system bus, and zero or more session buses.
+* Any process may connect to the system bus. The system bus limits which can own
+  names or send method calls, and only processes running as privileged users can
+  receive unicast messages not addressed to them. Every process may receive
+  broadcasts.
+* Each session bus has an owner (a user). Only its owner may connect; on
+  general-purpose Linux, a session bus is not treated as a privilege boundary,
+  so there is no further privilege separation between processes on it.
+
 Full coverage of securing D-Bus services is beyond the scope of this guide,
 however there are some steps which you can take when designing an API to ease
 security policy implementation.
 
-D-Bus security policies use an allow/deny model, where each message (method
-call, signal emission, etc.) can be allowed or denied according to the sum of
-all policy rules which match it. When designing an API, bear in mind the need to
-write such a security policy, and consider splitting up methods or providing
-more restricted versions which accept constrained parameters, so that they
-can be exposed with less restrictive security policies if needed by less trusted
-clients.
-
-Secondly, the default D-Bus security policy is restrictive enough to allow
-sensitive data, such as passwords, to be safely sent over the bus in unicast
-messages (including unicast signals); so there is no need to complicate APIs by
-implementing extra security. Note, however, that sensitive data must $em(not) be
-sent in broadcast signals, as they can be seen by all peers on the bus.
+D-Bus security policies are written as XML files in
+$file($var($$(sysconfdir$)/dbus-1/system.d)) and
+$file($var($$(sysconfdir$)/dbus-1/session.d)) and use an allow/deny model, where
+each message (method call, signal emission, etc.) can be allowed or denied
+according to the sum of all policy rules which match it. Each $code(<allow>) or
+$code(<deny>) rule in the policy should have the $code(own),
+$code(send_destination) or $code(receive_sender) attribute set.
+
+When designing an API, bear in mind the need to write and install such a
+security policy, and consider splitting up methods or providing more restricted
+versions which accept constrained parameters, so that they can be exposed with
+less restrictive security policies if needed by less trusted clients.
+
+Secondly, the default D-Bus security policy for the system bus is restrictive
+enough to allow sensitive data, such as passwords, to be safely sent over the
+bus in unicast messages (including unicast signals); so there is no need to
+complicate APIs by implementing extra security. Note, however, that sensitive
+data must $em(not) be sent in broadcast signals, as they can be seen by all
+peers on the bus. The default policy for the session bus is not restrictive, but
+it is typically not a security boundary.
 
 == Debugging
    [id="debugging"]
@@ -771,12 +852,12 @@ printing all messages it sees. The messages may be filtered using a standard
 $link[>>http://dbus.freedesktop.org/doc/dbus-specification.html#message-bus-routing-match-rules](D-Bus
 match rule) to make the stream more manageable.
 
-Previous versions of D-Bus have required the bus’ security policy to be manually
-relaxed to allow eavesdropping on all messages, especially on the system bus.
-This meant that debugging the system bus was difficult and insecure. The latest
-versions of D-Bus add support for monitor-only connections for the root user,
-which means that $cmd(dbus-monitor) can be run as root to painlessly monitor all
-messages on the system bus without modifying its security policy.
+Previous versions of D-Bus have required the security policy for the system bus
+to be manually relaxed to allow eavesdropping on all messages. This meant that
+debugging it was difficult and insecure. The latest versions of D-Bus add
+support for monitor-only connections for the root user, which means that
+$cmd(dbus-monitor) can be run as root to painlessly monitor all messages on the
+system bus without modifying its security policy.
 
 === Bustle
     [id="bustle"]
-- 
1.9.3