This is an update of an older blog post and is now taking advantage of the Python canlib package v1.7.
Developer Blog
20/02/2017
by Magnus Carlsson
Converting to plain ASCII
This is the second post in a 3-part series about using the Converter Library (kvlclib) in CANlib SDK:
- Writer formats and properties (1 of 3)
- Converting to plain ASCII (2 of 3)
- Special converter conditions (3 of 3)
The first part of this blog series looked into converter formats and properties, we will now continue and convert events from a given log file.
Let us assume that we have a log file, mylog.kme50, that we would like to convert to Plain text (.txt) format. The log file was created either by using the Kvaser Memorator Config Tool, or by using the kvmlib as described in an earlier blog post.1
Starting with the last code example we wrote in the previous blog article in this series, we now add setting up the name and format of our input file at the end.
# set up formats
out_fmt = kvlclib.WriterFormat(kvlclib.FileFormat.PLAIN_ASC)
in_fmt = kvlclib.ReaderFormat(kvlclib.FileFormat.KME50)
# set resulting output file name taking advantage of the extension
# defined in the format.
out_file = "myresult." + out_fmt.extension
print("Output filename is '%s'" % out_file)
# create converter
cnv = kvlclib.Converter(out_file, out_fmt)
# Set input file and format
cnv.setInputFile(INPUT_FILE, kvlclib.FileFormat.KME50)Let us also add a loop that converts the events one by one until we reach the end of our input file. The last thing we do is to call Converter.flush() in order to flush output to disk.
# Convert events from input file one by one until EOF is reached
while True:
try:
cnv.convertEvent()
except kvlclib.KvlcEndOfFile:
break
# Force a flush so that the resulting file is written to disk.
cnv.flush()Our first try at writing a kme50 converter program now looks as follows.
# 04_first_try_converting.py
from canlib import kvlclib
INPUT_FILE = 'mylog.kme50'
# set up formats
out_fmt = kvlclib.WriterFormat(kvlclib.FileFormat.PLAIN_ASC)
in_fmt = kvlclib.ReaderFormat(kvlclib.FileFormat.KME50)
# set resulting output file name taking advantage of the extension
# defined in the format.
out_file = "myresult." + out_fmt.extension
print("Output filename is '%s'" % out_file)
# create converter
cnv = kvlclib.Converter(out_file, out_fmt)
# Set input file and format
cnv.setInputFile(INPUT_FILE, kvlclib.FileFormat.KME50)
# Convert events from input file one by one until EOF is reached
while True:
try:
cnv.convertEvent()
except kvlclib.KvlcEndOfFile:
break
# Force a flush so that the resulting file is written to disk.
cnv.flush()Running the above code, assuming everything works as expected, the file myresult.txt is created. But running the same code again results in an error:
canlib.kvlclib.exceptions.KvlcGeneralError: Output file already exists (-6)Each converter has a number of properties that effect their inner workings. One of these properties is OVERWRITE which defaults to ‘0’, and that is the reason the code failed when running the second time – the resulting file already existed and the converter was not allowed to overwrite it.
As we saw in the previous blog post, we can ask if a property is supported by a specific format. So let us create a function that checks if a given property is supported. If the property is supported – and a value was supplied to the function – the function sets the property to the given value. We also take the opportunity to print the default value of the property to the console.
def trySetProperty(cnv, property, value=None):
# Check if the format supports the given property
if cnv.format.isPropertySupported(property):
# If a value is specified, set the property to this value
if value is not None:
cnv.setProperty(property, value)
# Get the property's default value
default = cnv.format.getPropertyDefault(property)
print(' %s is supported (Default: %s)' %
(property, default))
# Get the property's current value
value = cnv.getProperty(property)
print(' Current value: %s' % value)
else:
print(' %s is not supported' % property)Now let us use the trySetProperty() function and set the property OVERWRITE to ‘1’, we also limit the output to only the first channel by setting the CHANNEL_MASK property. Another useful property for the Plain text format is WRITE_HEADER which adds a nice header to the output file, so let us set that property as well. The final property we set is LIMIT_DATA_BYTES which controls how many data bytes to print. In the ASCII format this directly affects how wide our data table will be, and since we do not have any CAN FD2 messages in our input log, we set this property to 8 (bytes).
# allow output file to overwrite existing files
trySetProperty(cnv, kvlclib.Property.OVERWRITE, 1)
# we are only interested in the first channel
cnv.setProperty(kvlclib.Property.CHANNEL_MASK, 1)
# add nice header to the output file
trySetProperty(cnv, kvlclib.Property.WRITE_HEADER, 1)
# we are converting CAN traffic with max 8 bytes, so we can minimize
# the width of the data output to 8 bytes
trySetProperty(cnv, kvlclib.Property.LIMIT_DATA_BYTES, 8)Collecting all our improvements so far, we end up with the following program.
# 05_convert_kme.py
from canlib import kvlclib
def trySetProperty(cnv, property, value=None):
# Check if the format supports the given property
if cnv.format.isPropertySupported(property):
# If a value is specified, set the property to this value
if value is not None:
cnv.setProperty(property, value)
# Get the property's default value
default = cnv.format.getPropertyDefault(property)
print(' %s is supported (Default: %s)' %
(property, default))
# Get the property's current value
value = cnv.getProperty(property)
print(' Current value: %s' % value)
else:
print(' %s is not supported' % property)
# set up formats
out_fmt = kvlclib.WriterFormat(kvlclib.FileFormat.PLAIN_ASC)
in_fmt = kvlclib.ReaderFormat(kvlclib.FileFormat.KME50)
# set resulting output file name taking advantage of the extension
# defined in the format.
out_file = "myresult." + out_fmt.extension
print("Output filename is '%s'" % out_file)
# create converter
cnv = kvlclib.Converter(out_file, out_fmt)
# Set input file and format
cnv.setInputFile(INPUT_FILE, kvlclib.FileFormat.KME50)
# allow output file to overwrite existing files
trySetProperty(cnv, kvlclib.Property.OVERWRITE, 1)
# we are only interested in the first channel
cnv.setProperty(kvlclib.Property.CHANNEL_MASK, 1)
# add nice header to the output file
trySetProperty(cnv, kvlclib.Property.WRITE_HEADER, 1)
# we are converting CAN traffic with max 8 bytes, so we can minimize
# the width of the data output to 8 bytes
trySetProperty(cnv, kvlclib.Property.LIMIT_DATA_BYTES, 8)
# Convert events from input file one by one until EOF is reached
while True:
try:
cnv.convertEvent()
except kvlclib.KvlcEndOfFile:
break
# Force a flush so that the resulting file is written to disk.
cnv.flush()Running the full program results in the following output.
Kvaser Memorator Log
====================
Converted from Memorator Binary format at: 2018-08-14 07:47:10
Settings:
Format of data field: DEC
Format of id field: DEC
Timestamp Offset: 0.000000 s
CAN channel: 1
Time Chan Identifier Flags DLC Data Counter
====================================================================================
0.242162 Trigger (type=0x1, active=0x00, pre-trigger=0, post-trigger=-1)
32.003407 1 501 Rx 8 41 23 0 0 0 0 0 0 2
32.006923 1 501 Rx 8 234 0 0 0 0 0 0 0 4
32.007795 1 504 Rx 8 51 179 79 67 0 0 0 0 6
32.008373 1 402 Rx 8 51 51 134 66 0 0 0 0 8
32.009127 1 402 Rx 8 51 51 243 64 0 0 0 0 10
32.009925 1 504 Rx 8 0 0 56 66 0 0 0 0 12
32.010670 1 503 Rx 8 0 0 52 67 0 0 0 0 14
32.011413 1 504 Rx 8 154 25 90 67 0 0 0 0 16
32.012167 1 402 Rx 8 205 204 140 64 0 0 0 0 18
32.015623 1 504 Rx 8 205 204 68 65 0 0 0 0 20This ends the second blog article in which we created a simple program that used the Kvaser Converter library, kvlclib, to convert a kme50 log file into a plain text file. In the last article of this series, we will look at some special conditions that can arise when converting a log file.
If you have any questions, comments or suggestion for future blog articles, you can contact us directly via mail at support@kvaser.com.
Original Article
This is the second post in a 3-part series about using the Converter Library (kvlclib) in CANlib SDK:
- Writer formats and properties (1 of 3)
- Converting to plain ASCII (2 of 3)
- Special converter conditions (3 of 3)
The first part of this blog series looked into converter formats and properties, we will now continue and convert events from a given log file.
Let us assume that we have a log file, mylog.kme50, that we would like to convert to Plain text (.txt) format. The log file was created either by using the Kvaser Memorator Config Tool, or by using the kvmlib as described in an earlier blog post.[1]
Starting with the latest code sample we wrote in the first part of this blog series, we now add some code for setting up the format and filename of our input file at the end.
import canlib.kvlclib as kvlc
# set output format
fmt = kvlc.WriterFormat(kvlc.FILE_FORMAT_PLAIN_ASC)
# the name of the formatter is fetched using kvlcGetWriterName() internally
print("Output format is '%s'" % fmt.name)
# set resulting output filename taking advantage of the extension defined in
# the format. (Uses kvlcGetWriterExtension() under the hood.)
outfile = "myresult." + fmt.extension
print("Output filename is '%s'" % outfile)
# create converter
kc = kvlc.Kvlclib(outfile, fmt)
# Set input filename and format
inputfile = "mylog.kme50"
print("Input filename is '%s'" % inputfile)
kc.setInputFile(inputfile, file_format=kvlc.FILE_FORMAT_KME50)
Let us now add a loop that converts the events one by one until we reach the end of our input file.
# Convert events from input file one by one until EOF is reached
while True:
try:
kc.convertEvent()
except kvlc.KvlcEndOfFile:
breakThe last thing we need to do is to call kvlcDeleteConverter() in order to flush output to disk and free memory.
# Delete converter, flush result to disk and free memory
kc.deleteConverter()Running the above code once, the file myresult.txt is created. But running the same code again results in an error:
File "c:\dev\python\pycanlib\src\canlib\kvlclib.py", line 459, in convertEvent
self.dll.kvlcConvertEvent(self.handle)
File "c:\dev\python\pycanlib\src\canlib\kvlclib.py", line 408, in _kvlc_err_check
raise KvlcError(self, result)
canlib.kvlclib.KvlcError: [KvlcError] convertEvent: Output file already exists (-6)Each converter has a number of properties that effect their inner workings. One of these properties is KVLC_PROPERTY_OVERWRITE which defaults to ’0’, and that is the reason the code failed when running the second time – the resulting file already existed and the converter was not allowed to overwrite the existing file.
As we saw in the previous blog post, we can ask if a property is supported by a specific format. So let us create a function that checks if a given property is supported. If the property is supported – and a value was supplied to the function – the function sets the property to the given value. We also take the opportunity to print the default value of the property to the console.
def trySetProperty(converter, property, value=None):
# Check if the format supports the given property
if converter.format.isPropertySupported(property):
# If a value was specified, set the property to this value
if value is not None:
converter.setProperty(property, value)
# get the property's default value
default = converter.getPropertyDefault(property)
print(" PROPERTY_%s is supported (Default: %s)" %
(property['name'], default))
# get the property's current value
value = converter.getProperty(property)
print(" Current value: %s" % value)
else:
print(" PROPERTY %s is not supported" %
(property['name']))
Listing 4: Defining a function that tries to set a given property.
Now let us use the trySetProperty() function defined in Listing 4 and set the KVLC_PROPERTY_OVERWRITE to ‘1’. A second useful property for the Plain text format is KVLC_PROPERTY_WRITE_HEADER which adds a nice header to the output file so let us set that property as well. The final property we set is KVLC_PROPERTY_LIMIT_DATA_BYTES which controls how many data bytes to print. In the ASCII format this directly affects how wide our data table will be, and since we do not have any CAN FD[2] messages in our input log, we set this property to 8 (bytes).
# allow output file to overwrite existing files
trySetProperty(kc, kvlc.PROPERTY_OVERWRITE, 1)
# add nice header to the output file
trySetProperty(kc, kvlc.PROPERTY_WRITE_HEADER, 1)
# we are converting CAN traffic with max 8 bytes, so we can minimize the width
# of the data output to 8 bytes
trySetProperty(kc, kvlc.PROPERTY_LIMIT_DATA_BYTES, 8)Listing 5: Setting some convenient properties when using the plain text writer.
Collecting all our improvements so far, we end up with the program shown in Listing 6.
import canlib.kvlclib as kvlc
def trySetProperty(converter, property, value=None):
# Check if the format supports the given property
if converter.format.isPropertySupported(property):
# If a value was specified, set the property to this value
if value is not None:
converter.setProperty(property, value)
# get the property's default value
default = converter.getPropertyDefault(property)
print(" PROPERTY_%s is supported (Default: %s)" %
(property['name'], default))
# get the property's current value
value = converter.getProperty(property)
print(" Current value: %s" % value)
else:
print(" PROPERTY %s is not supported" %
(property['name']))
# set output format
fmt = kvlc.WriterFormat(kvlc.FILE_FORMAT_PLAIN_ASC)
# the name of the formatter is fetched using kvlcGetWriterName() internally
print("Output format is '%s'" % fmt.name)
# set resulting output filename taking advantage of the extension defined in
# the format. (Uses kvlcGetWriterExtension() under the hood.)
outfile = "myresult." + fmt.extension
print("Output filename is '%s'" % outfile)
# create converter
kc = kvlc.Kvlclib(outfile, fmt)
# Set input filename and format
inputfile = "mylog.kme50"
print("Input filename is '%s'" % inputfile)
kc.setInputFile(inputfile, file_format=kvlc.FILE_FORMAT_KME50)
# allow output file to overwrite existing files
trySetProperty(kc, kvlc.PROPERTY_OVERWRITE, 1)
# add nice header to the output file
trySetProperty(kc, kvlc.PROPERTY_WRITE_HEADER, 1)
# we are converting CAN traffic with max 8 bytes, so we can minimize the width
# of the data output to 8 bytes
trySetProperty(kc, kvlc.PROPERTY_LIMIT_DATA_BYTES, 8)
# Convert events from input file one by one until EOF is reached
while True:
try:
kc.convertEvent()
except kvlc.KvlcEndOfFile:
break
# Delete converter, flush result to disk and free memory
kc.deleteConverter()Listing 6: Simple program to convert a kme50 log file to plain text output.
Our second attempt as shown in Listing 6 can be run multiple times, and the top part of the generated file myresult.txt is shown in Listing 7.
Kvaser Memorator Log
====================
Converted from Memorator Binary format at: 1/11/2017 15:54:02
Settings:
Format of data field: DEC
Format of id field: DEC
Timestamp Offset: 0.000000000 s
CAN channel: 1 2
Time Chan Identifier Flags DLC Data Counter
=============================================================================================
0.247909525 Trigger (type=0x1, active=0x00, pre-trigger=0, post-trigger=-1)
10.648166375 1 55 Rx 1 2 1
10.648166375 2 55 Rx 1 2 2
12.464742375 1 340 Rx 8 0 0 0 0 0 0 0 0 3
12.464742375 2 340 Rx 8 0 0 0 0 0 0 0 0 4Listing 7: Top rows of the plain text output result from the conversion.
In this article, we created a simple program to convert a kme50 log file to plain text. In the last article of this series, we will look at the special conditions that can arise when converting a log file.
If you have any questions, comments or suggestion for future blog articles, you can contact us directly at support@kvaser.com.
Footnotes
[1] Reading from a Kvaser Memorator and saving the logged data using kvmlib was done in www.kvaser.com/developer-blog/digging-deeper-into-kvmlib
[2] CAN FD allows data fields up to 64 bytes per frame. See the blog article www.kvaser.com/developer-blog/can-fd for more information about CAN FD.
