Network Working Group P. Barker
Request for Comments: 1564 University College London
Category: Informational R. Hedberg
Technical University Delft
January 1994
This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Distribution of this memo is unlimited.
This document defines a set of criteria by which a DSA implementation may be judged. Particular issues covered include conformance to standards; performance; demonstrated interoperability. The intention is that the replies to the questions posed provide a fairly full description of a DSA. Some of the questions will yield answers which are purely descriptive; others, however, are intended to elicit answers which give some measure of the utility of the DSA. The marks awarded for a DSA in each particular area should give a good indication of the DSA's capabilities, and its suitability for particular uses.
Please send comments to the authors or to the discussion group
<osi-ds@CS.UCL.AC.UK>.
1. Overview 2 2. General Information 3 3. Conformance to OSI Standards 43.1 Directory protocols
4. Other protocols 7 5. Extensions to the 1988 Standard 75.1 Schema
6. Miscellaneous characteristics 10
7. Management tools 117.1 Dynamic system management
8. Operational Use 12 9. Interoperability 12 10. Performance 1310.1 Speed for various operations
11. Security Considerations 21 12. Authors' Addresses 21
The purpose of this document is to define some metrics by which DSA products can be measured. Such metrics are valuable as whilst an X.500 DSA must conform to the specification in the standard - this is a sine qua non - protocol conformance is not in itself the hallmark of a usable implementation. A DSA must perform operations within a reasonable time; a DSA must offer good throughput of queries; a DSA must be able to handle a reasonable volume of data; if modification operations are provided, some sort of access control must be provided; a DSA and its data must be manageable.
In many respects, it is almost impossible to say that one DSA is better than other from looking at the responses to questions in this document. For some, the cost or level of support will be the key criterion. For another user, the flexibility of the schema management facilities, or the feasibility of running the DSA over an existing relational database, will be of prime importance. In many respects DSAs will just be different, rather than better or worse. However, all other things being equal, the look-up speed of a DSA is very obviously measurable, and there is a substantial number of questions on the speed of the various X.500 operations, and in particular on the look-up operations.
Throughout this document, some of the questions posed are annotated with a square-bracketed points score and an explanation as to how the points should be allocated. For example, a question might be
appended with "[2 if yes]", indicating score 2 points for an affirmative answer to that question. These points scores should be collated in Table 1 at the end of the document. The questions on DSA performance are judged to be important enough to have a separate table for those results: they appear in Table 2 (and optionally Table 3). Together, these tables constitute a measure of the DSA.
The metrics are on a section by section basis, which should help the
reader who is seeking, for example, a DSA with fast look-up
capabilities and extensive access control facilities, to focus on the
critical aspects of a DSA for their particular requirement. No
conclusions should be inferred from adding the scores together into
one overall grand total and comparing such totals for different DSAs,
as no attempt is made to assign weights to the different
characteristics.
Whilst much of this document should usually be completed by the developers or suppliers of an implementation, the section on performance could be completed by anyone running the implementation. Indeed, it will be beneficial if several sets of performance figures can be gathered for each implementation, for a variety of hardware platforms.
This section contains general information about the implementation under discussion.
1. Name of the information provider
3. Version number of the DSA described in this document
5. Name and address of supplier or person to contact
(a) Hardware (If appropriate, can summarise as, for example "generic UNIX platform") ..................................
(b) O/S (state version if critical)
ii. VMS) [1] ................................................
iii. MS-DOS [1] ..............................................
iv. Macintosh [1] ...........................................
(a) Read ASE? [2] ...............................................
(b) Search ASE? [2] .............................................
(c) Modify ASE? [2] .............................................
(a) Chained read ASE? [2] .......................................
(b) Chained search ASE? [2] .....................................
(c) Chained modify ASE? [2] .....................................
(a) Supported application-contexts? ............................
(b) Capable of acting as first-level DSA? [1] ...................
(c) Chained mode supported? [1] ................................
(d) Security-level(s) supported? [1 for strong + 1 for protected simple + 1 for simple authentication] .......................
(e) All attribute types according to X.520? [1] ................
(f) All object classes according to X.521? [1] .................
(a) Abstract syntaxes of application contexts ...................
(b) Abstract syntaxes of information framework ..................
(c) Minimal knowledge ...........................................
(d) Support of root context .....................................
(e) Abstract syntax - attribute types ...........................
(f) Abstract syntax - object classes ............................
Dynamic requirements [2 if yes on all]
(a) Mapping onto underlying services ............................
(b) Distributed operations - referrals ..........................
(c) DirectoryAccessAC - referrals ...............................
(d) DirectorySystemAC - referrals ...............................
(e) Chained mode ................................................
Does the DSA conform to the following implementors' agreements? If so, state parts and version numbers.
Does the DSA conform to the following profiles? If so, state which version numbers.
State any other GOSIP profiles to which the DSA conforms ............
(a) TP.x over CONS (state transport class)? [2] ................
(b) TP.4 over CLNS? [2] .........................................
(c) TP.x over X.25(1980) (state transport class)? [2] ..........
(a) Enforce this hierarchy? ....................................
(b) Allow the enforcement of this hierarchy? ...................
24. Are structure rules optional or mandatory?
(a) TP.0 over RFC1006 over TCP/IP [3] ...........................
(b) State any other options supported. .........................
(a) Attribute types with existing syntaxes? With compilation [1], or without compilation [2] .............................
(b) Attribute syntaxes? With compilation [1], or without compilation [2] .............................................
(c) Attribute sets? With compilation [1], or without compilation
[2] .........................................................
(d) Object classes? With compilation [1], or without compilation
[2] .........................................................
34. Does the DSA support any other replication mechanisms?
(a) Replication part of RFC1276 [2] .............................
(b) Other (please give a reference to any description of the
mechanisms, and indicate whether these mechanisms are used by
any other implementations) [1 for any mechanism] ............
(a) Replication of a single entry? [2] .........................
(b) Replication of a set of sibling entries? [2] ...............
(c) Replication of a subtree? [2] ..............................
[2] .............................................................
(a) Allow a user to maintain their own entry? [1] ..............
(b) Allow a user to maintain some attributes in their own entry, but not all attributes? [1] ................................
(c) Give management rights to a DSA manager in a fashion analogous to the privileges given to a UNIX super-user? [1] ..........
(d) Give management rights to a data manager on a per subtree basis? [1] .................................................
(e) Give management rights (to an entry, group of entries, subtree, etc) to a group of users? [1] .....................
(f) Give access rights to users on the basis of the leading portion of their Distinguished Name? [1] ...................
(g) Is it possible to define a protection mechanism for each individual attribute type in one entry? [1] ................
(h) Maximum number of Distinguished Names that can be defined for one access right to one attribute in one entry? If unlimited, state the constraints. [1 if more than 6 DNs are feasible] :
(i) Does the DSA support the extended access control techniques
described in "An Access Control approach for Searching and
Listing" by Hardcastle-Kille and Howes, in the Internet
Draft, OSI-DS 21? [2]
(j) If there are features of the access control mechanisms which
are not brought out by the above questions, please describe
these additional features [up to 2 for wonderful additional
features!] .................................................
40. If the DSA uses RFC1006 and/or X.25(1980) at the network layer,
does the DSA conform to RFC1277, "Encoding Network Addresses to
support operation over non-OSI lower layers" [3] ...............
47. Does the DSA make of indexing? [2 if yes]
If so:
(a) Can the database be fully inverted? [1] .................... If not, state for which:
i. attributes indexes are automatically built
ii. attributes/attribute syntaxes indexes may be built ......
(b) Does the index improve performance on:
ii. Leading substring match [1] .............................
iii. Approximate match [1] ...................................
iv. Any substring match [1] .................................
v. Trailing substring match [1]
(c) What is the increase in run-time size of an entry when adding
an index?
(d) What is the increase in on-disk database size of adding
another index?
(a) DAP? [1] ....................................................
(b) CMIP? [1] ...................................................
(c) SNMP? [1] ...................................................
51. Are there tools for controlling a run-time DSA? [2]
The DSA may have lots of wonderful features -- on paper! But has the DSA been shown to work in practice? The following measures are intended to give some measure of confidence that the DSA's viability has been demonstrated.
57. What is the largest set of DSAs supporting an organisation?
The X.500 Directory is the OSI Directory. OSI stands for Open Systems Interconnection -- DSAs have to be able to inter-operate. They also have to be seen to interoperate.
60. Is this DSA in use in X.500 pilots?
(a) Is this DSA in use anywhere in the COSINE/Internet Pilot? [3]
(b) Is this DSA in use in any other major pilot? [2] ...........
This section should give an outline to the expected performance of the DSA. A number of operations are timed in order to give a feel for the DSA's speed and throughput. Note that all operations should be resolvable within a single DSA. Chaining and referral are not assessed, although it should be possible to infer, albeit approximately, the speed of distributed operations.
Create an organisational DSA with 20000 entries below the organisation node. Sub-divide this data into a number of organisational units, one of which should contain 1000 entries, another of which should contain 100 entries, and a third which should contain just 10 entries. The entries, which should differ, should be created with the following attributes:
(a) Common Name
(b) Surname
(c) Telephone number
(d) Postal Address (of 100 characters)
(e) Object class
ii. In all the tests, two timings should be taken. In order to normalise the test results as much as possible, it is suggested that these tests be undertaken on an otherwise lightly loaded machine.
(a) A typical "cold start" reading should be given. In this case the system will not have the advantage of any benefits that derive from operating system paging, or caching.
(b) A best possible figure should be given, which indicates the upper limit of DSA performance.
iii. The timings should relate to the default set-up, and should be
entered in Table 2. If significant performance gains can be made
by use of configuration options, such as building extra indexes
to support searches, measures of the improved performance may
also be given, and should be entered in Table 3.
Attention should be also drawn to any optimisations, heuristic or
otherwise, which are not evidenced in the following tests.
iv. Please note that the tests should be made using a DUA and DSA with full 7-layer stacks, rather than some lightweight protocol.
The tests are described, one subsection per operation. The results should be entered in Table 2 (and Table 3 if a non-default set-up is also measured).
The time it takes for a DUA to bind to the Directory. This time should include all the initialisation time a DUA process needs before it can query the Directory: e.g., reading of tailor files, schema information, etc. Give the bind time for each of the following levels of authentication. State "n/a" if the implementation does not support a particular level of authentication.
Give the time for listing a set of organisational unit sibling entries.
In this section, two sets of search operations should be performed on the DSA.
ii. An organisation subtree search, on the subtree of 20000 entries.
The following searches should be tried. Unless otherwise stated, the "XXX" or "YYY" part of the search filter should be chosen in such a way as to return a single result. Unless stated otherwise the results should return all attributes for the entry.
surname=XXX
commonName=XXX*
commonName=*XXX*
commonName=*XXX
commonName"=XXX
objectClass=person AND
(commonName=XXX* OR telephoneNumber=*YYY)
objectClass=*
In this case, no attribute values should be returned in the result set.
(a) 0 children
(b) 10 children
(c) 1000 children
Modify an attribute value, other than an RDN value, for an entry which has
(a) Add description attribute
(b) Remove description attribute
Modify an RDN value for an entry with the following number of siblings.
(a) 10 siblings
(b) 1000 siblings
As the time taken for a single read will usually be negligible, the following list and set of reads should give a clearer indication of the query rate.
The results of the tests just described should be entered in Table 2 (and optionally Table 3), at the end of the document.
Date of test......................................................... Name of tester ...................................................... The results will be directly correlated to the test set-up used, and in particular, the hardware. Please answer the following questions to describe the test environment:
(a) Processor (make and model) ..................................
(b) Processor speed (MIPS) ......................................
(c) Primary memory available ....................................
(d) If disk-based DSA, disk I/O interface and disk speed ........
(e) O/S version .................................................
(f) Network type and bandwidth (e.g., 10 Mbit Ethernet) .........
(g) Protocols in transport layer and below (e.g., TP 0, RFC1006, TCP/IP) .....................................................
(h) How/where timings obtained?
+-------------------------------------------------+
| Section || Points |
+--------------------------------||---------------+
| No.||Description |Maximum|Scored |
+----||--------------------------|-------|--------+
| || | | |
| 2||General Information | 20 | |
+----||--------------------------|-------|--------+
| || | | |
| 3||Conformance to OSI | 35 | |
+----||--------------------------|-------|--------+
| || | | | |
| 4||Other protocols | 5 | |
+----||--------------------------|-------|--------+
| || | | | |
| 5||Extensions| Schema | 16 | |
+----|| |---------------|-------|--------+
| || | | | |
| ||to the |Replication | 10 | |
+----|| |---------------|-------|--------+
| || | | | |
| ||1988 |Access Control | 15 | |
+----|| |---------------|-------|--------+
| || | | | |
| ||standard |Miscellaneous | 5 | |
+----||--------------------------|-------|--------+
| ||Miscellaneous | | |
| 6||characteristics | 15 | |
+----||--------------------------|-------|--------+
| || | | |
| 7||Management tools | 10 | |
+----||--------------------------|-------|--------+
| || | | |
| 8||Operational use | 10 | |
+----||--------------------------|-------|--------+
| || | | |
| 9||Interoperability | 10 | |
+----||--------------------------|-------|--------+
| || | see | |
| 10||Performance |table 2| |
+-------------------------------------------------+
Table 1: DSA Metrics
+------------------------------------------------------+
| Operation || Cold DSA || Optimum |
| || || Performance |
+-------------------||---------------||----------------+
| Bind || || |
| --Anonymous ||.............. || .............. |
| --Simple ||.............. || .............. |
| --Simple Prot ||.............. || .............. |
| --Strong ||.............. || .............. |
+-------------------||---------------||----------------+
| List || || |
| -- 10 entries ||.............. || .............. |
| -- 1000 entries||.............. || .............. |
+-------------------||---------------||----------------+
| Search |single|subtree |single|subtree |
| |level | |level | |
| |------|--------|------|----------|
| -- exact |.... |...... |..... | ...... |
| -- leading sub |.... |...... |..... | ...... |
| -- any sub |.... |...... |..... | ...... |
| -- trailing sub |.... |...... |..... | ...... |
| -- approx |.... |...... |..... | ...... |
| -- complex |.... |...... |..... | ...... |
| -- return all |.... |...... |..... | ...... |
+--------------------|------|--------|------|----------|
| Read ||.............. || .............. |
+-------------------||---------------||----------------+
| Add || || |
| 0 siblings ||.............. || .............. |
| 10 siblings ||.............. || .............. |
| 1000 siblings ||.............. || .............. |
+-------------------||---------------||----------------+
| Modify || || |
| 10 siblings ||.............. || .............. |
| 1000 siblings ||.............. || .............. |
+-------------------||---------------||----------------+
| Modify RDN || || |
| 10 siblings ||.............. || .............. |
| 1000 siblings ||.............. || .............. |
+-------------------||---------------||----------------+
| Query rate ||.............. || .............. |
+-------------------||---------------||----------------+
Table 2: Speed of operations - default set-up
+------------------------------------------------------+
| Operation || Cold DSA || Optimum |
| || || Performance |
+-------------------||---------------||----------------+
| Bind || || |
| --Anonymous ||.............. || .............. |
| --Simple ||.............. || .............. |
| --Simple Prot ||.............. || .............. |
| --Strong ||.............. || .............. |
+-------------------||---------------||----------------+
| List || || |
| -- 10 entries ||.............. || .............. |
| -- 1000 entries||.............. || .............. |
+-------------------||---------------||----------------+
| Search |single|subtree |single|subtree |
| |level | |level | |
| |------|--------|------|----------|
| -- exact |.... |...... |..... | ...... |
| -- leading sub |.... |...... |..... | ...... |
| -- any sub |.... |...... |..... | ...... |
| -- trailing sub |.... |...... |..... | ...... |
| -- approx |.... |...... |..... | ...... |
| -- complex |.... |...... |..... | ...... |
| -- return all |.... |...... |..... | ...... |
+--------------------|------|--------|------|----------|
| Read ||.............. || .............. |
+-------------------||---------------||----------------+
| Add || || |
| 0 siblings ||.............. || .............. |
| 10 siblings ||.............. || .............. |
| 1000 siblings ||.............. || .............. |
+-------------------||---------------||----------------+
| Modify || || |
| 10 siblings ||.............. || .............. |
| 1000 siblings ||.............. || .............. |
+-------------------||---------------||----------------+
| Modify RDN || || |
| 10 siblings ||.............. || .............. |
| 1000 siblings ||.............. || .............. |
+-------------------||---------------||----------------+
| Query rate ||.............. || .............. |
+-------------------||---------------||----------------+
Table 3: Speed of operations - non-default set-up
Security issues are not discussed in this memo.
Paul Barker
Department of Computer Science
University College London
Gower Street
London
WC1E 6BT
United Kingdom
Phone: +44 71 380 7366
Fax: +44 71 387 1397 EMail: P.Barker@cs.ucl.ac.uk
Roland Hedberg
Rekencentrum
Delft Technical University
Michiel de Ruyterweg 10-12
Postbus 354, 2600 AJ Delft
The Netherlands
Phone: +31 15 785210
EMail: Roland.Hedberg@rc.tudelft.nl
OR
Roland Hedberg
Umdac
University of Umea
s-901 87 Umea
Sweden
Phone: +46 90 165204
EMail: Roland.Hedberg@umdac.umu.se