streamsx.toolkits package¶
Utility functions for IBM Streams topology applications¶
For details of implementing applications in Python for IBM Streams including IBM Cloud Pak for Data and the Streaming Analytics service running on IBM Cloud see:
Overview¶
Provides provides some utility functions to work with SPL toolkits, for example for use in a Python Notebook.
-
streamsx.toolkits.
create_keystore
(client_cert, client_private_key, store_filepath, store_passwd=None)¶ Creates a JKS type keystore for a client certificate and its private RSA key.
The aliases of the certificate in the keystore will be ‘client_cert’. The private key will be encrypted with the store password.
Example:
client_cert = """ -----BEGIN CERTIFICATE----- MIICtjCCAZ4CCQCBrBBIDMiWTTANBgkqhkiG9w0BAQsFADAqMRMwEQYDVQQKDApp by5zdHJpbXppMRMwEQYDVQQDDApjbGllbnRzLWNhMB4XDTE5MDgxNjA5MzAzOVoX DTIwMDgwNjA5MzAzOVowEDEOMAwGA1UEAwwFcm9sZWYwggEiMA0GCSqGSIb3DQEB AQUAA4IBDwAwggEKAoIBAQC4rmdUhW0ZE0O7mwaYilIWb8Lqy/g4G7zYKa+5nVNI pM8Y9DLM6UmXFmO5SyiYWNzJCvbrj1u20deOrLetTpid6/LactoekVF6Gwkcy7uh FvkMvvzO00zU/7HGdzMRqwIWHZFVFQ7eNJcF5ITTPSyjfuXKCqVlTN70U6pVR1Kt jvh95U4qi6UxfrhfPSgX5rL1UgmMnYemeFWW9AmROk5FNOrXpiH9pRM9e034J/Lx X4fSArMc/HB2qmFkat5RSYNdYE47Upv/ufE0bl1CXvZ4hlKdQ2nyFV0ZkZ/IdtNV wXRczXmVKKEvCISTZWri1qFG2zs5ZibQXt2TWi/Hmb7rAgMBAAEwDQYJKoZIhvcN AQELBQADggEBABhDMkkXfGXagGY6EgZ4cev1h18jVm0uNknlUw6UXgLfoX7OBUmr yjfYWIqdNOewY3IZ1iTQHoEWXuFmuEUalb+CfYXdoZuHx/s211sLBy0S9pod3h+f VcwovrXqfE0V/Of5+3svE2p+O2N+UschjH/Ha9ljJ4CbVN0b/K8KkOt2dd6lMn2l jLyzjCBtnxBYnS1tbyILK6Mdb0Q6EeYBQw5K6x+f9gRvcyNpBcMMFZFk0n0Hr+zW 4MuwSg7eM4OU1hExqUSILEfaVtsj4IgCimUN5RGEYBp8YFzT0GYnEm+c4iaEb4Uh 1aDgpwpHly2t0dI7m3i2VYN4/UeRi2tXtS8= -----END CERTIFICATE----- """ private_key = """ -----BEGIN PRIVATE KEY----- MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQC4rmdUhW0ZE0O7 mwaYilIWb8Lqy/g4G7zYKa+5nVNIpM8Y9DLM6UmXFmO5SyiYWNzJCvbrj1u20deO rLetTpid6/LactoekVF6Gwkcy7uhFvkMvvzO00zU/7HGdzMRqwIWHZFVFQ7eNJcF 5ITTPSyjfuXKCqVlTN70U6pVR1Ktjvh95U4qi6UxfrhfPSgX5rL1UgmMnYemeFWW 9AmROk5FNOrXpiH9pRM9e034J/LxX4fSArMc/HB2qmFkat5RSYNdYE47Upv/ufE0 bl1CXvZ4hlKdQ2nyFV0ZkZ/IdtNVwXRczXmVKKEvCISTZWri1qFG2zs5ZibQXt2T Wi/Hmb7rAgMBAAECggEBAJyes1zXprUcRFXm3AojTAhhEaGEB2a2B0oe2DsGtCmo M0XLWwWyVkl+oGX02mGRAvSwisUsl7AMd800prSgMgfDoPonatl/jeSK5wh3sxCj ZeSsV8OjKMZ+biqMzk+Ogj91JdpM+IXzfZxut0JZ3/Br3r6glrG5Jl94ULzvtqse YBXzi+9wbDjhzUrZea8GBSEgm4er8nGVlvl3cGA6B05KCkndSrpzJmD/1O1yHl2w jWcxnWbMjgJ+ZgTkWxNx1AACP1XnUubuHtH5X6CdyyexJVIcvSou29LE4AEgskzf N5f9g4WQqm/wqJouA6g2O2bYuH6RqPBR6LvRiWN+XCkCgYEA9W+xqD3EadT+9bkq SFkAHntFMnkmiu78azr5CSqXGZWiT12BloZg44Y4ouHkz0Q7UIOkuKnt8XUT+XTx cMJXdM/L8A7XadbLRGm+jQkPkamxzbhNha3RPqNcMmuSl+bus0/Y4CPlRzqmVFM2 6i+7AJdSovt7ELPyB5VRoFTjqM8CgYEAwKFJ5C0DPbtm8lwf42Vg9lKReR2lhrdh 70rIvuBwLpvmDEjT4E5XlCQZvmwCJhYQLeR7pdWTZUIUY3PRg6saUPt6mseQaY5G UMfd8eKNmYDq1dzw66L/rXVfLXL/U965mlchgrv1aYxFvS2oaXc06FGWf7HiX70r AYL3s4NsVyUCgYAntVFt4chkQvSY9sMzf0q4o8L3PoiQ08d85caiyAQ4PHBNHNi5 bLZCOVXybE+jhOt/xDyjL8LyXLnFkzj4wEh7+p6JES2izwCF6gHNOZkOPQEqQlYw eYWv96gT9Dl8X/1gf1ucL9KJaqC8ZdYtn1lG6DSTb9JiSmqu9WaGZRWMGwKBgQCz Y4DUpbam/qbYfUKVGgw6WJpxGFLWKmFh31sjpw/R2A1c01W9Rjf5FcWDuEIoBdP7 aRRdU5B4dBlYpLswy18um4OKMN8j7gM9p2PhpwIHOi94CtGyH5HLj8qbIMT27iwa t5nVjgT1zlQD4Uk9Lm5vC8wEDzypvBkYx0a2PmXeoQKBgQCdoz1xPj5Ltrocn992 ci3mu3a5OxozeJVi36XSoWwr9HrK0arTaTEkbqcQxnjuCx/pIxM+FQQrVVjpAdop N1lLtfM0/X9cXohp2CwQGFldjOd7pF0t28ewN0wRij6oOdVa/u3EsVqijnhSEhOl YmR4uwOY4uG/ecALiF/2BBz5OA== -----END PRIVATE KEY----- """ key_store_password = create_keystore(client_cert, private_key, "/tmp/keystore.jks") # read key and certificate from existing files store_passwd = "Y3456tugzh" create_keystore("/tmp/client_cert.pem", "/tmp/client_priv_key.pem", "/tmp/keystore.jks", store_passwd)
- Parameters
client_cert (str) – the filename with a client certificate in PEM format or the certificate itself
client_private_key (str) – the filename with a private key in PEM format or the private key itself
store_filepath (str) – the filename of the keystore file. The location must be writable. Intermediate directories are not created.
store_passwd (str) – the password for the private key and the keystore. When
None
, a 16 characters password is generated.
- Returns
the generated or given password of the keystore and the private key
- Return type
str
See also:
create_truststore()
,extend_keystore()
New in version 1.1.
-
streamsx.toolkits.
extend_keystore
(client_cert, client_private_key, store_filepath, store_passwd)¶ Extends a JKS type keystore with a client certificate and its private RSA key.
The alias of the certificate in the keystore will be client_cert-<n>. The private key will be encrypted with the store password.
Example:
store_file = '/tmp/keystore.jks' store_passwd = create_keystore(client_cert='/tmp/db_client.crt', client_private_key='/tmp/db_client.key', store_filepath=store_file) extend_keystore(client_cert='/tmp/mqtt_client.crt', client_private_key='/tmp/mqtt_client.key', store_filepath=store_file, store_passwd=store_passwd)
- Parameters
client_cert (str) – the filename with a client certificate in PEM format or the certificate itself
client_private_key (str) – the filename with a private key in PEM format or the private key itself
store_filepath (str) – the filename of the keystore file. The file must exist and be writable.
store_passwd (str) – the password for the keystore. It is also used to encrypt the given private key.
- Returns
the aliases of the added entries (currently a list with one element)
- Return type
list
See also:
create_keystore()
New in version 1.2.
-
streamsx.toolkits.
create_truststore
(trusted_cert, store_filepath, store_passwd=None)¶ Creates a JKS type keystore from a single trusted certificate or a list of trusted certificates. A keystore with trusted certificates is therefore referred to as a truststore. Truststores are typically used to store root certificates or self-signed certificates that servers present to clients. The aliases of the certificates in the keystore will be ‘ca_cert-i’, where i is a number starting at 0.
Example:
trusted_cert = """ -----BEGIN CERTIFICATE----- MIIDJzCCAg+gAwIBAgIJAJhu1pO2qbj7MA0GCSqGSIb3DQEBCwUAMCoxEzARBgNV BAoMCmlvLnN0cmltemkxEzARBgNVBAMMCmNsdXN0ZXItY2EwHhcNMTkwODE2MDc1 ODI3WhcNMjAwODE1MDc1ODI3WjAqMRMwEQYDVQQKDAppby5zdHJpbXppMRMwEQYD VQQDDApjbHVzdGVyLWNhMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA poC8jaqI9izS96CS39m4aOFruWDPxcrDsfU4gIbFveMHOV57F/LWkiC4GHWrcujj YHEKShMyP2rMfmQp148lxEZMc84zigB89ZUjWiFFR8PZBeX7syUmvTb/sEwWswOP epMUwqoNwDJ49HLQKbrSuZ+T4XymcPAmHU+Osm77hpJRGsloN1uVCBwzJJgxf9oz m0qLDMI4sw9GD3B8gjPZowZ70LtDnKkmG3hGXZwW1cvL2vuECdKWm77FUe+L2NVx c6cnZ5htNX0wjATf5hTN6uXsNUK/Xf6lKwcRgatYQVGT+kd+tDraHwhkID8cPyFi F+vbIhDzhKSBHGwYIGn+SwIDAQABo1AwTjAdBgNVHQ4EFgQUIYRTcQDd8rskiN2Q cV101xbmI9MwHwYDVR0jBBgwFoAUIYRTcQDd8rskiN2QcV101xbmI9MwDAYDVR0T BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAQtuxhqeT0z5fNB3Vs13TvVLfXPiG YSTwYvxfkrxGKtDiLjIZ6Q6LJjMcWLG4x3I0WmGgvytTs04S4B+1vp721jmqRKm9 xCJ+Obi/eWmN4uu1rr0fR7fXr7wWLFBiDr8dZX3EnzjWeVWeNESoZTpaMKScuYTZ ocAT5iL3ZDUj/lwqJRptmzGFcdko+woFae68HRx1ygSgROls7bXy/CwgME0LFFQp B+2YAhUw1sPU410JUxU3/v6R5vJfI9imE75aha3U7UbeOX8+1+Cu3HOT1QMn80k2 6LnZeMCCgCBp+Yz3YNeUMRejMU6x4WlhTPO7bBq3tKGgwCoyGIX25wMM1Q== -----END CERTIFICATE----- """ generated_truststore_password = create_truststore(trusted_cert, "/tmp/truststore.jks") # multiple certificates in existing files store_passwd = "Y3456tugzh" create_truststore(["/tmp/cert1.pem", "/tmp/cert2.pem"], "/tmp/truststore.jks", store_passwd)
- Parameters
trusted_cert (list|str) – a list of filenames with trusted certificates in PEM format or the certificates literally
store_filepath (str) – the filename of the truststore file. The location must be writable. Intermediate directories are not created.
store_passwd (str) – the password for the keystore. When
None
, a 16 characters password is generated.
- Returns
the generated or given password of the truststore
- Return type
str
See also:
create_keystore()
,extend_truststore()
New in version 1.1.
-
streamsx.toolkits.
extend_truststore
(trusted_cert, store_filepath, store_passwd)¶ Extends an existing truststore by trusted certificate entries.
Example:
store_file = '/tmp/truststore.jks' store_passwd = create_truststore('/tmp/db.crt', store_filepath=store_file) # add two more certificates: extend_truststore(['/tmp/mqtt.crt', '/tmp/kafka.crt'], store_filepath=store_file, store_passwd=store_passwd)
- Parameters
trusted_cert (list|str) – a list of filenames with trusted certificates in PEM format or the certificates literally
store_filepath (str) – the filename of the truststore file. The truststore must exist and be writable.
store_passwd (str) – the password for the given truststore.
- Returns
the aliases of the added entries
- Return type
list
See also:
create_truststore()
New in version 1.2.
-
streamsx.toolkits.
download_toolkit
(toolkit_name, repository_name=None, url=None, target_dir=None)¶ Downloads the latest SPL toolkit from GitHub for the given toolkit name.
Example for adding the com.ibm.streams.nlp toolkit with latest toolkit from GitHub:
import streamsx.toolkits as tkutils # download toolkit from GitHub location = tkutils.download_toolkit('com.ibm.streamsx.nlp') # add toolkit to topology streamsx.spl.toolkit.add_toolkit(topo, location)
- Parameters
toolkit_name (str) – the toolkit directory in the archive (where toolkit.xml is located), for example “com.ibm.streamsx.nlp”
repository_name (str) – name of the GitHub repository at “github.com/IBMStreams”, for example “streamsx.nlp”. Set this parameter if repository name is not part of toolkit name without “com.ibm.”.
url (str) – the download URL, apply link to toolkit archive (*.tgz) to be downloaded. ‘https://github.com/IBMStreams/REPOSITORY-NAME/releases/latest’ is used per default.
target_dir (str) – the directory where the toolkit is unpacked to. If a relative path is given, the target_dir is appended to the system temporary directory, for example to /tmp on Unix/Linux systems.
- Returns
the absolute path of the toolkit directory
- Return type
str
Note
This function requires an outgoing Internet connection
-
streamsx.toolkits.
get_pypi_packages
(package_name=None)¶ Discover the latest Python packages available on pypi.org.
- Parameters
package_name (str) – the name of the Python package to be searched for. If
None
is given,function searches for all available streamsx Python packages. (the) –
- Returns
A dictionary with mappings from Python package name to the package version
- Return type
dict
Note
This function requires an outgoing Internet connection
-
streamsx.toolkits.
get_installed_packages
()¶ Discover installed streamsx python packages in your Python environment.
- Returns
A dictionary with mappings from Python package name to the package version
- Return type
dict
-
streamsx.toolkits.
get_build_service_toolkits
(streams_cfg=None, verify=False)¶ Discover toolkits on IBM Streams build service.
- Parameters
streams_cfg (dict) – Service instance details of the IBM Streams instance. If value is None then the external build service endpoint is used.
verify (bool) – Set to True if SSL verification shall be enabled.
Example, use the function in IBM Cloud Pak for Data with service instance details retrieved from icpd_util.get_service_details:
from icpd_core import ipcd_util cfg = icpd_util.get_service_details(name='instanceName', instance_type='streams') import streamsx.toolkits as tkutils build_service_toolkits = tkutils.get_build_service_toolkits(cfg) print(build_service_toolkits)
- Returns
A dictionary with mappings from toolkit name to the toolkit version
- Return type
dict
-
streamsx.toolkits.
get_github_toolkits
(toolkit_name=None)¶ Discover the latest releases of product toolkits available on public GitHub.
- Parameters
toolkit_name (str) – the name of the toolkit to be searched for. If
None
is given,function searches for all product toolkits. (the) –
- Returns
A dictionary with mappings from toolkit name to the toolkit version
- Return type
dict
Note
This function requires an outgoing Internet connection