generic-library/webrtc
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

First version of PythonCharts.

Browse Source 
The reason why it is so simple is that I wanted to get something into the project that people can use to compare different test runs easily. More functionality will come later.

tools/python_charts/src/gviz_api.py is a copy of the Google visualization Python API available from http://google-visualization-python.googlecode.com/svn/trunk/

Review URL: http://webrtc-codereview.appspot.com/257003

git-svn-id: http://webrtc.googlecode.com/svn/trunk@893 4adac7df-926f-26a2-2b94-8c16560cd09d
This commit is contained in:
kjellander@webrtc.org 2011-11-07 15:25:47 +00:00
parent b353d21560
commit 689cb300b5
10 changed files with 1649 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
tools/python_charts/README Normal file
View File

@ -0,0 +1,41 @@
This file describes how to setup Eclipse and then the Python Charts project
Setup Eclipse
-------------
These instructions were tested on Linux, but are very similar for Windows and
Mac.
1. Ensure you have Python 2.x installed
2. Download and install Google App Engine SDK for Python from 
http://code.google.com/appengine/downloads.html
3. Note which location you put App Engine in, as this will be needed later on.
4. Download Eclipse from http://www.eclipse.org. Any distribution will probably
do, but if you're going to do mainly web development, you might pick Eclipse
IDE for JavaScript Web Developers
5. Install the PyDev plugin using the Eclipse update site mentioned at 
http://pydev.org/download.html
6. Install the Google Plugin for Eclipse: http://code.google.com/eclipse/
Setup the project
-----------------
Generic instructions are available at
http://code.google.com/appengine/docs/python/gettingstarted/ but the following
should be enough:
1. Launch Eclipse and create a workspace
2. Create a new PyDev Project
3. In the PyDev Project wizard, uncheck the "Use Default" checkbox for Project
contents and browse to your tools/python_charts directory.
4. Enter a project name. We'll assume PythonCharts in the examples below.
5. In the radio button of the lower part of the window, select
"Add project directory to the PYTHONPATH"
6. Click Finish
7. Select the Run > Run Configuration… menu item
8. Create a new "Python Run" configuration
9. Select your Python Charts project as project
10. As Main Module, enter the path to your dev_appserver.py, which is a part
of your App Engine installation,
e.g. /usr/local/google_appengine/dev_appserver.py
11. At the Arguments tab, enter the location of your project root.
Using Eclipse variables if your project name is PythonCharts:
${workspace_loc:PythonCharts}
12. Launch the development app server by clicking the Run button.
13. Launch a browser and go to http://localhost:8080

9
tools/python_charts/app.yaml Normal file
View File

@ -0,0 +1,9 @@
application: webrtc-python-charts
version: 1
runtime: python
api_version: 1
handlers:
- url: /*
script: webrtc/main.py

49
tools/python_charts/data/vp8_hw.py Normal file
View File

@ -0,0 +1,49 @@
# Sample output from the video_quality_measurment program, included only for
# reference. Geneate your own by running with the --python flag and then change
# the filenames in main.py
test_configuration = [{'name': 'name', 'value': 'Quality test'},
{'name': 'description', 'value': ''},
{'name': 'test_number', 'value': '0'},
{'name': 'input_filename', 'value': 'foreman_cif.yuv'},
{'name': 'output_filename', 'value': 'foreman_cif_out.yuv'},
{'name': 'output_dir', 'value': '.'},
{'name': 'packet_size_in_bytes', 'value': '1500'},
{'name': 'max_payload_size_in_bytes', 'value': '1440'},
{'name': 'packet_loss_mode', 'value': 'Uniform'},
{'name': 'packet_loss_probability', 'value': '0.000000'},
{'name': 'packet_loss_burst_length', 'value': '1'},
{'name': 'exclude_frame_types', 'value': 'ExcludeOnlyFirstKeyFrame'},
{'name': 'frame_length_in_bytes', 'value': '152064'},
{'name': 'use_single_core', 'value': 'False'},
{'name': 'keyframe_interval;', 'value': '0'},
{'name': 'video_codec_type', 'value': 'VP8'},
{'name': 'width', 'value': '352'},
{'name': 'height', 'value': '288'},
{'name': 'bit_rate_in_kbps', 'value': '500'},
]
frame_data_types = {'frame_number': ('number', 'Frame number'),
'encoding_successful': ('boolean', 'Encoding successful?'),
'decoding_successful': ('boolean', 'Decoding successful?'),
'encode_time': ('number', 'Encode time (us)'),
'decode_time': ('number', 'Decode time (us)'),
'encode_return_code': ('number', 'Encode return code'),
'decode_return_code': ('number', 'Decode return code'),
'bit_rate': ('number', 'Bit rate (kbps)'),
'encoded_frame_length': ('number', 'Encoded frame length (bytes)'),
'frame_type': ('string', 'Frame type'),
'packets_dropped': ('number', 'Packets dropped'),
'total_packets': ('number', 'Total packets'),
'ssim': ('number', 'SSIM'),
'psnr': ('number', 'PSNR (dB)'),
}
frame_data = [{'frame_number': 0, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 94676, 'decode_time': 37942, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 1098, 'encoded_frame_length': 4579, 'frame_type': 'Other', 'packets_dropped': 0, 'total_packets': 4, 'ssim': 0.910364, 'psnr': 35.067258},
{'frame_number': 1, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 244007, 'decode_time': 39421, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 306, 'encoded_frame_length': 1277, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.911859, 'psnr': 35.115193},
{'frame_number': 2, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 240508, 'decode_time': 38918, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 330, 'encoded_frame_length': 1379, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.913597, 'psnr': 35.181604},
{'frame_number': 3, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 243449, 'decode_time': 39664, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 298, 'encoded_frame_length': 1242, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.912378, 'psnr': 35.164710},
{'frame_number': 4, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 248024, 'decode_time': 39115, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 332, 'encoded_frame_length': 1385, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.911471, 'psnr': 35.109488},
{'frame_number': 5, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 246910, 'decode_time': 39146, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 416, 'encoded_frame_length': 1734, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 2, 'ssim': 0.915231, 'psnr': 35.392300},
{'frame_number': 6, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 242953, 'decode_time': 38827, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 279, 'encoded_frame_length': 1165, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.916130, 'psnr': 35.452889},
{'frame_number': 7, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 247343, 'decode_time': 41429, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 393, 'encoded_frame_length': 1639, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 2, 'ssim': 0.919356, 'psnr': 35.647128},
{'frame_number': 8, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 249529, 'decode_time': 40329, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 487, 'encoded_frame_length': 2033, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 2, 'ssim': 0.924705, 'psnr': 36.179837},
{'frame_number': 9, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 249408, 'decode_time': 41716, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 583, 'encoded_frame_length': 2433, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 2, 'ssim': 0.928433, 'psnr': 36.589875},
]

49
tools/python_charts/data/vp8_sw.py Normal file
View File

@ -0,0 +1,49 @@
# Sample output from the video_quality_measurment program, included only for
# reference. Geneate your own by running with the --python flag and then change
# the filenames in main.py
test_configuration = [{'name': 'name', 'value': 'Quality test'},
{'name': 'description', 'value': ''},
{'name': 'test_number', 'value': '0'},
{'name': 'input_filename', 'value': 'foreman_cif.yuv'},
{'name': 'output_filename', 'value': 'foreman_cif_out.yuv'},
{'name': 'output_dir', 'value': '.'},
{'name': 'packet_size_in_bytes', 'value': '1500'},
{'name': 'max_payload_size_in_bytes', 'value': '1440'},
{'name': 'packet_loss_mode', 'value': 'Uniform'},
{'name': 'packet_loss_probability', 'value': '0.000000'},
{'name': 'packet_loss_burst_length', 'value': '1'},
{'name': 'exclude_frame_types', 'value': 'ExcludeOnlyFirstKeyFrame'},
{'name': 'frame_length_in_bytes', 'value': '152064'},
{'name': 'use_single_core', 'value': 'False'},
{'name': 'keyframe_interval;', 'value': '0'},
{'name': 'video_codec_type', 'value': 'VP8'},
{'name': 'width', 'value': '352'},
{'name': 'height', 'value': '288'},
{'name': 'bit_rate_in_kbps', 'value': '500'},
]
frame_data_types = {'frame_number': ('number', 'Frame number'),
'encoding_successful': ('boolean', 'Encoding successful?'),
'decoding_successful': ('boolean', 'Decoding successful?'),
'encode_time': ('number', 'Encode time (us)'),
'decode_time': ('number', 'Decode time (us)'),
'encode_return_code': ('number', 'Encode return code'),
'decode_return_code': ('number', 'Decode return code'),
'bit_rate': ('number', 'Bit rate (kbps)'),
'encoded_frame_length': ('number', 'Encoded frame length (bytes)'),
'frame_type': ('string', 'Frame type'),
'packets_dropped': ('number', 'Packets dropped'),
'total_packets': ('number', 'Total packets'),
'ssim': ('number', 'SSIM'),
'psnr': ('number', 'PSNR (dB)'),
}
frame_data = [{'frame_number': 0, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 12427, 'decode_time': 4403, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 2270, 'encoded_frame_length': 9459, 'frame_type': 'Other', 'packets_dropped': 0, 'total_packets': 7, 'ssim': 0.947050, 'psnr': 38.332820},
{'frame_number': 1, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 3292, 'decode_time': 821, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 88, 'encoded_frame_length': 368, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.927272, 'psnr': 35.883510},
{'frame_number': 2, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 4295, 'decode_time': 902, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 130, 'encoded_frame_length': 544, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.920539, 'psnr': 35.457107},
{'frame_number': 3, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 3880, 'decode_time': 767, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 171, 'encoded_frame_length': 714, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.917434, 'psnr': 35.389298},
{'frame_number': 4, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 4471, 'decode_time': 909, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 248, 'encoded_frame_length': 1035, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.918892, 'psnr': 35.570229},
{'frame_number': 5, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 4447, 'decode_time': 976, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 269, 'encoded_frame_length': 1123, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.920609, 'psnr': 35.769663},
{'frame_number': 6, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 4432, 'decode_time': 891, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 271, 'encoded_frame_length': 1132, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 1, 'ssim': 0.922672, 'psnr': 35.913519},
{'frame_number': 7, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 5026, 'decode_time': 1068, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 366, 'encoded_frame_length': 1529, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 2, 'ssim': 0.925505, 'psnr': 36.246713},
{'frame_number': 8, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 4877, 'decode_time': 1051, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 369, 'encoded_frame_length': 1538, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 2, 'ssim': 0.926122, 'psnr': 36.305984},
{'frame_number': 9, 'encoding_successful': True , 'decoding_successful': True , 'encode_time': 4712, 'decode_time': 1087, 'encode_return_code': 0, 'decode_return_code': 0, 'bit_rate': 406, 'encoded_frame_length': 1692, 'frame_type': 'Delta', 'packets_dropped': 0, 'total_packets': 2, 'ssim': 0.927183, 'psnr': 36.379735},
]

1048
tools/python_charts/gviz_api.py Executable file
View File

File diff suppressed because it is too large Load Diff

80
tools/python_charts/templates/chart_page_template.html Normal file
View File

@ -0,0 +1,80 @@
<html>
<!--
Copyright (c) 2011 The WebRTC project authors. All Rights Reserved.
Use of this source code is governed by a BSD-style license
that can be found in the LICENSE file in the root of the source
tree. An additional intellectual property rights grant can be found
in the file PATENTS. All contributing project authors may
be found in the AUTHORS file in the root of the source tree.
Template file to be used to generate Charts for Video Quality Metrics.
-->
<head>
<link href="http://code.google.com/css/codesite.pack.04102009.css"
rel="stylesheet" type="text/css" />
</head>
<script src="https://www.google.com/jsapi" type="text/javascript"></script>
<script>
google.load('visualization', '1', {packages:['table', 'corechart']});
google.setOnLoadCallback(drawTable);
function drawTable() {
/* Build data table and views */
var ssim_data_table =
new google.visualization.DataTable(%(json_ssim_data)s);
var psnr_data_table =
new google.visualization.DataTable(%(json_psnr_data)s);
var packet_loss_data_table =
new google.visualization.DataTable(%(json_packet_loss_data)s);
var bit_rate_data_table =
new google.visualization.DataTable(%(json_bit_rate_data)s);
/* Display tables and charts */
var ssim_chart = new google.visualization.LineChart(
document.getElementById('table_div_ssim'));
ssim_chart.draw(ssim_data_table, {
colors: ['blue', 'orange'],
vAxis: {title: 'SSIM'},
hAxis: {title: 'Frame'},
width: 1200, height: 300,
});
var psnr_chart = new google.visualization.LineChart(
document.getElementById('table_div_psnr'));
psnr_chart.draw(psnr_data_table, {
colors: ['blue', 'orange'],
vAxis: {title: 'PSNR(dB)'},
hAxis: {title: 'Frame'},
width: 1200, height: 300,
});
var packet_loss_chart = new google.visualization.LineChart(
document.getElementById('table_div_packet_loss'));
packet_loss_chart.draw(packet_loss_data_table, {
colors: ['blue', 'orange'],
vAxis: {title: 'Packets dropped'},
hAxis: {title: 'Frame'},
width: 1200, height: 300,
});
var bit_rate_chart = new google.visualization.LineChart(
document.getElementById('table_div_bit_rate'));
bit_rate_chart.draw(bit_rate_data_table, {
colors: ['blue', 'orange', 'red'],
vAxis: {title: 'Bit rate'},
hAxis: {title: 'Frame'},
width: 1200, height: 300,
});
}
</script>
<body>
<h3>Messages:</h3>
<pre>%(messages)s</pre>
<h3>Metrics measured per frame:</h3>
<div id="table_div_ssim"></div>
<div id="table_div_psnr"></div>
<div id="table_div_packet_loss"></div>
<div id="table_div_bit_rate"></div>
</body>
</html>

8
tools/python_charts/webrtc/__init__.py Normal file
View File

@ -0,0 +1,8 @@
#!/usr/bin/env python
# Copyright (c) 2011 The WebRTC project authors. All Rights Reserved.
#
# Use of this source code is governed by a BSD-style license
# that can be found in the LICENSE file in the root of the source
# tree. An additional intellectual property rights grant can be found
# in the file PATENTS. All contributing project authors may
# be found in the AUTHORS file in the root of the source tree.

134
tools/python_charts/webrtc/data_helper.py Normal file
View File

@ -0,0 +1,134 @@
#!/usr/bin/env python
# Copyright (c) 2011 The WebRTC project authors. All Rights Reserved.
#
# Use of this source code is governed by a BSD-style license
# that can be found in the LICENSE file in the root of the source
# tree. An additional intellectual property rights grant can be found
# in the file PATENTS. All contributing project authors may
# be found in the AUTHORS file in the root of the source tree.
__author__ = 'kjellander@webrtc.org (Henrik Kjellander)'
class DataHelper(object):
"""
Helper class for managing table data.
This class does not verify the consistency of the data tables sent into it.
"""
def __init__(self, data_list, table_description, names_list, messages):
""" Initializes the DataHelper with data.
Args:
data_list: List of one or more data lists in the format that the
Google Visualization Python API expects (list of dictionaries, one
per row of data). See the gviz_api.DataTable documentation for more
info.
table_description: dictionary describing the data types of all
columns in the data lists, as defined in the gviz_api.DataTable
documentation.
names_list: List of strings of what we're going to name the data
columns after. Usually different runs of data collection.
messages: List of strings we might append error messages to.
"""
self.data_list = data_list
self.table_description = table_description
self.names_list = names_list
self.messages = messages
self.number_of_datasets = len(data_list)
self.number_of_frames = len(data_list[0])
def CreateData(self, field_name, start_frame=0, end_frame=0):
""" Creates a data structure for a specified data field.
Creates a data structure (data type description dictionary and a list
of data dictionaries) to be used with the Google Visualization Python
API. The frame_number column is always present and one column per data
set is added and its field name is suffixed by _N where N is the number
of the data set (0, 1, 2...)
Args:
field_name: String name of the field, must be present in the data
structure this DataHelper was created with.
start_frame: Frame number to start at (zero indexed). Default: 0.
end_frame: Frame number to be the last frame. If zero all frames
will be included. Default: 0.
Returns:
A tuple containing:
- a dictionary describing the columns in the data result_data_table below.
This description uses the name for each data set specified by
names_list.
Example with two data sets named 'Foreman' and 'Crew':
{
'frame_number': ('number', 'Frame number'),
'ssim_0': ('number', 'Foreman'),
'ssim_1': ('number', 'Crew'),
}
- a list containing dictionaries (one per row) with the frame_number
column and one column of the specified field_name column per data
set.
Example with two data sets named 'Foreman' and 'Crew':
[
{'frame_number': 0, 'ssim_0': 0.98, 'ssim_1': 0.77 },
{'frame_number': 1, 'ssim_0': 0.81, 'ssim_1': 0.53 },
]
"""
# Build dictionary that describes the data types
result_table_description = {'frame_number': ('string', 'Frame number')}
for dataset_index in range(self.number_of_datasets):
column_name = '%s_%s' % (field_name, dataset_index)
column_type = self.table_description[field_name][0]
column_description = self.names_list[dataset_index]
result_table_description[column_name] = (column_type, column_description)
# Build data table of all the data
result_data_table = []
# We're going to have one dictionary per row.
# Create that and copy frame_number values from the first data set
for source_row in self.data_list[0]:
row_dict = { 'frame_number': source_row['frame_number'] }
result_data_table.append(row_dict)
# Pick target field data points from the all data tables
if end_frame == 0: # Default to all frames
end_frame = self.number_of_frames
for dataset_index in range(self.number_of_datasets):
for row_number in range(start_frame, end_frame):
column_name = '%s_%s' % (field_name, dataset_index)
# Stop if any of the data sets are missing the frame
try:
result_data_table[row_number][column_name] = \
self.data_list[dataset_index][row_number][field_name]
except IndexError:
self.messages.append("Couldn't find frame data for row %d "
"for %s" % (row_number, self.names_list[dataset_index]))
break
return (result_table_description, result_data_table)
def GetOrdering(self, table_description):
""" Creates a list of column names, ordered alphabetically except for the
frame_number column which always will be the first column.
Args:
table_description: A dictionary of column definitions as defined by the
gviz_api.DataTable documentation.
Returns:
A list of column names, where frame_number is the first and the
remaining columns are sorted alphabetically.
"""
# The JSON data representation generated from gviz_api.DataTable.ToJSon()
# must have frame_number as its first column in order for the chart to
# use it as it's X-axis value series.
# gviz_api.DataTable orders the columns by name by default, which will
# be incorrect if we have column names that are sorted before frame_number
# in our data table.
columns_ordering = ['frame_number']
# add all other columns:
for column in sorted(table_description.keys()):
if column != 'frame_number':
columns_ordering.append(column)
return columns_ordering

92
tools/python_charts/webrtc/data_helper_test.py Normal file
View File

@ -0,0 +1,92 @@
#!/usr/bin/env python
# Copyright (c) 2011 The WebRTC project authors. All Rights Reserved.
#
# Use of this source code is governed by a BSD-style license
# that can be found in the LICENSE file in the root of the source
# tree. An additional intellectual property rights grant can be found
# in the file PATENTS. All contributing project authors may
# be found in the AUTHORS file in the root of the source tree.
__author__ = 'kjellander@webrtc.org (Henrik Kjellander)'
import unittest
import webrtc.data_helper
class Test(unittest.TestCase):
def setUp(self):
# Simulate frame data from two different test runs, with 2 frames each.
self.frame_data_0 = [{'frame_number': 0, 'ssim': 0.5, 'psnr': 30.5},
{'frame_number': 1, 'ssim': 0.55, 'psnr': 30.55}]
self.frame_data_1 = [{'frame_number': 0, 'ssim': 0.6, 'psnr': 30.6},
{'frame_number': 0, 'ssim': 0.66, 'psnr': 30.66}]
self.all_data = [ self.frame_data_0, self.frame_data_1 ]
# Test with frame_number column in a non-first position sice we need to
# support reordering that to be able to use the gviz_api as we want.
self.type_description = {
'ssim': ('number', 'SSIM'),
'frame_number': ('number', 'Frame number'),
'psnr': ('number', 'PSRN'),
}
self.names = ["Test 0", "Test 1"]
def testCreateData(self):
messages = []
helper = webrtc.data_helper.DataHelper(self.all_data, self.type_description,
self.names, messages)
description, data_table = helper.CreateData('ssim')
self.assertEqual(3, len(description))
self.assertTrue('frame_number' in description)
self.assertTrue('ssim_0' in description)
self.assertTrue('number' in description['ssim_0'][0])
self.assertTrue('Test 0' in description['ssim_0'][1])
self.assertTrue('ssim_1' in description)
self.assertTrue('number' in description['ssim_1'][0])
self.assertTrue('Test 1' in description['ssim_1'][1])
self.assertEqual(0, len(messages))
self.assertEquals(2, len(data_table))
row = data_table[0]
self.assertEquals(0, row['frame_number'])
self.assertEquals(0.5, row['ssim_0'])
self.assertEquals(0.6, row['ssim_1'])
row = data_table[1]
self.assertEquals(1, row['frame_number'])
self.assertEquals(0.55, row['ssim_0'])
self.assertEquals(0.66, row['ssim_1'])
description, data_table = helper.CreateData('psnr')
self.assertEqual(3, len(description))
self.assertTrue('frame_number' in description)
self.assertTrue('psnr_0' in description)
self.assertTrue('psnr_1' in description)
self.assertEqual(0, len(messages))
self.assertEquals(2, len(data_table))
row = data_table[0]
self.assertEquals(0, row['frame_number'])
self.assertEquals(30.5, row['psnr_0'])
self.assertEquals(30.6, row['psnr_1'])
row = data_table[1]
self.assertEquals(1, row['frame_number'])
self.assertEquals(30.55, row['psnr_0'])
self.assertEquals(30.66, row['psnr_1'])
def testGetOrdering(self):
""" Tests that the ordering help method returns a list with frame_number
first and the rest sorted alphabetically """
messages = []
helper = webrtc.data_helper.DataHelper(self.all_data, self.type_description,
self.names, messages)
description, data_table = helper.CreateData('ssim')
columns = helper.GetOrdering(description)
self.assertEqual(3, len(columns))
self.assertEqual(0, len(messages))
self.assertEqual('frame_number', columns[0])
self.assertEqual('ssim_0', columns[1])
self.assertEqual('ssim_1', columns[2])
if __name__ == "__main__":
unittest.main()

139
tools/python_charts/webrtc/main.py Normal file
View File

@ -0,0 +1,139 @@
#!/usr/bin/env python
# Copyright (c) 2011 The WebRTC project authors. All Rights Reserved.
#
# Use of this source code is governed by a BSD-style license
# that can be found in the LICENSE file in the root of the source
# tree. An additional intellectual property rights grant can be found
# in the file PATENTS. All contributing project authors may
# be found in the AUTHORS file in the root of the source tree.
__author__ = 'kjellander@webrtc.org (Henrik Kjellander)'
import os
import gviz_api
import webrtc.data_helper
def main():
"""
This Python script displays a web page with test created with the
video_quality_measurement program, which is a tool in WebRTC.
The script requires on two external files and one Python library:
- A HTML template file with layout and references to the json variables
defined in this script
- A data file in Python format, containing the following:
- test_configuration - a dictionary of test configuration names and values.
- frame_data_types - a dictionary that maps the different metrics to their
data types
- frame_data - a list of dictionaries where each dictionary maps a metric to
it's value.
- The gviz_api.py of the Google Visualization Python API, available at
http://code.google.com/p/google-visualization-python/
The HTML file is shipped with the script, while the data file must be
generated by running video_quality_measurement with the --python flag
specified.
"""
print 'Content-type: text/html\n' # the newline is required!
page_template_filename = '../templates/chart_page_template.html'
# The data files must be located in the project tree for app engine being
# able to access them.
data_filenames = [ '../data/vp8_sw.py', '../data/vp8_hw.py' ]
# Will contain info/error messages to be displayed on the resulting page.
messages = []
# Load the page HTML template.
try:
f = open(page_template_filename)
page_template = f.read()
f.close()
except IOError as e:
ShowErrorPage('Cannot open page template file: %s<br>Details: %s' %
(page_template_filename, e))
return
# Read data from external Python script files. First check that they exist.
for filename in data_filenames:
if not os.path.exists(filename):
messages.append('Cannot open data file: %s' % filename)
data_filenames.remove(filename)
# Read data from all existing input files.
data_list = []
test_configurations_list = []
names = []
for filename in data_filenames:
read_vars = {} # empty dictionary to load the data into.
execfile(filename, read_vars, read_vars)
test_configuration = read_vars['test_configuration']
table_description = read_vars['frame_data_types']
table_data = read_vars['frame_data']
# Verify the data in the file loaded properly.
if not table_description or not table_data:
messages.append('Invalid input file: %s. Missing description list or '
'data dictionary variables.', filename)
continue
# Frame numbers appear as number type in the data, but Chart API requires
# values of the X-axis to be of string type.
# Change the frame_number column data type:
table_description['frame_number'] = ('string', 'Frame number')
# Convert all the values to string types:
for row in table_data:
row['frame_number'] = str(row['frame_number'])
# Store the unique data from this file in the high level lists.
test_configurations_list.append(test_configuration)
data_list.append(table_data)
# Use the filenames for name; strip away directory path and extension.
names.append(filename[filename.rfind('/')+1:filename.rfind('.')])
# Create data helper and build data tables for each graph.
helper = webrtc.data_helper.DataHelper(data_list, table_description,
names, messages)
# Loading it into gviz_api.DataTable objects and create JSON strings.
description, data = helper.CreateData('ssim')
ssim = gviz_api.DataTable(description, data)
json_ssim_data = ssim.ToJSon(helper.GetOrdering(description))
description, data = helper.CreateData('psnr')
psnr = gviz_api.DataTable(description, data)
json_psnr_data = psnr.ToJSon(helper.GetOrdering(description))
description, data = helper.CreateData('packets_dropped')
packet_loss = gviz_api.DataTable(description, data)
json_packet_loss_data = packet_loss.ToJSon(helper.GetOrdering(description))
description, data = helper.CreateData('bit_rate')
# Add a column of data points for the desired bit rate to be plotted.
# (uses test configuration from the last data set, assuming it is the same
# for all of them)
desired_bit_rate = -1
for row in test_configuration:
if row['name'] == 'bit_rate_in_kbps':
desired_bit_rate = int(row['value'])
if desired_bit_rate == -1:
ShowErrorPage('Cannot find bit rate in the test configuration.')
return
# Add new column data type description.
description['desired_bit_rate'] = ('number', 'Desired bit rate (kbps)')
for row in data:
row['desired_bit_rate'] = desired_bit_rate
bit_rate = gviz_api.DataTable(description, data)
json_bit_rate_data = bit_rate.ToJSon(helper.GetOrdering(description))
# Format the messages list with newlines.
messages = '\n'.join(messages)
# Put the variables as JSon strings into the template.
print page_template % vars()
def ShowErrorPage(error_message):
print '<html><body>%s</body></html>' % error_message
if __name__ == '__main__':
main()