Tags: hildrum/PerfKitBenchmarker
Tags
0.23.0 Known Issues: * Resources fail to be cleaned up when PerfKitBenchmarker receives SIGINT while running with `--parallel` (GoogleCloudPlatformGH-529). New Features: * Added [CloudStack](http://cloudstack.apache.org) provider support (thanks @syed; GoogleCloudPlatformGH-558). * **❢ BREAKING CHANGE:** Added a more flexible benchmark configuration system (GoogleCloudPlatformGH-510, GoogleCloudPlatformGH-546). This supports running the same benchmark multiple times, and running a benchmark using heterogeneous machine types and clouds. See [the wiki](https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/wiki/PerfkitBenchmarker-Configurations) for more information. This replaces the configuration system under `perfkitbenchmarker/deployment`. * New benchmark: `redis_ycsb`, which runs YCSB against a single YCSB node (GoogleCloudPlatformGH-511). Enhancements: * Added flag: `--archive_bucket`, to archive the results of a run to Google Cloud Storage or S3 (GoogleCloudPlatformGH-489). * Added `--prefill_device` option to fio benchmark. Note that the `fio` benchmark now defaults to *not* pre-filling the device when testing against a filesystem (GoogleCloudPlatformGH-515, FH-516). * Updated `cloud_bigtable_ycsb` benchmark to pre-split table, following HBase recommendions (GoogleCloudPlatformGH-524). * Added additional logging to `LinuxVirtualMachine.RobustRemoteCommand` (GoogleCloudPlatformGH-534). * Added support for local disks with EC2 `d2` types. * Kubernetes: Support `emptyDisk` disk types (thanks @mateusz-blaszkowski GoogleCloudPlatformGH-565). * `cluster_boot`: Add `num_vms` metadata (GoogleCloudPlatformGH-575). * Add published `run_uri` to VM metadata (GoogleCloudPlatformGH-579). * Add a flag to specify the Azure Python library version in object storage benchmark, default to the latest version (GoogleCloudPlatformGH-585, GoogleCloudPlatformGH-587). Bugfixes and maintenance updates: * Kubernetes: Adapt provider to work with different versions of `kubectl` (Thanks @mateusz-blaszkowski; GoogleCloudPlatformGH-574). * Eliminated a spurious resource warning (GoogleCloudPlatformGH-522). * Fixed a string comparison bug in Object storage benchmark (GoogleCloudPlatformGH-526). * Rackspace; Removed an extra parameter from firewall implementation (thanks @meteorfox, GoogleCloudPlatformGH-531). * Rackspace: Fixed an SSH key upload bug (thanks @meteorfox, GoogleCloudPlatformGH-539). * Fix an issue with loggin errors in `vm_util.RunThreaded` (GoogleCloudPlatformGH-542; thanks @mateusz-blaszkowski for reporting). * `mongodb_ycsb`: Update default write concern to "Acknowledged" (equivalent to the previous deprecated option, safe) (GoogleCloudPlatformGH-543). * Specify SSH port when using SCP (thanks @mateusz-blaszkowski GoogleCloudPlatformGH-548). * Fix string conversion (thanks @mateusz-blaszkowski GoogleCloudPlatformGH-556). * `hbase_ycsb`: Make load records use a BufferedMutator (GoogleCloudPlatformGH-566).
Release v0.22.1 Changes: * Update the default image used in Azure tests to Ubuntu 14.04.3 (runs failed with the current image). (cherry-pick of GoogleCloudPlatform#520)
* New Features: * Add Kubernetes as a provider where benchmarks can be run (thanks * @mateusz-blaszkowski) (GoogleCloudPlatformGH-475) * Add EPFL CloudSuite WebSearch Workload (thanks @vukasin92) (GoogleCloudPlatformGH-422, * GoogleCloudPlatformGH-479) * Aerospike_yscb_benchmark (GoogleCloudPlatformGH-486) * Enhancements: * OpenStack checks/changes for attaching volumes, default volume size, * config improvements, … (thanks @kivio) (GoogleCloudPlatformGH-454, GoogleCloudPlatformGH-459, GoogleCloudPlatformGH-464) * Fio test improvements and clean-up - template files, logging, * ... (GoogleCloudPlatformGH-421, GoogleCloudPlatformGH-501, GoogleCloudPlatformGH-502) * Add improvement to use compiled crcmod when benchmarking Google * Cloud Storage (GoogleCloudPlatformGH-461) * Collect Fio data over time via “--run_for_minutes” flag (GoogleCloudPlatformGH-477) * Execute command improvements to add better logging and * parallelism (GoogleCloudPlatformGH-488, GoogleCloudPlatformGH-470) * Bugfixes and maintenance updates: * Update to allow PKB to work with the latest Azure CLI 0.9.9 (GoogleCloudPlatformGH-485) * Updated fio version to 2.2.10 (GoogleCloudPlatformGH-509) * fio iodepth flag fix “--io_depths” (GoogleCloudPlatformGH-495) * Fixes a race condition on AWS/Azure where networks in use were * deleted (GoogleCloudPlatformGH-506) * Added a note on how to fix dependencies when tox tests fail (GoogleCloudPlatformGH-505) * Fixed broken Apache project links (GoogleCloudPlatformGH-492, GoogleCloudPlatformGH-467) * Netperf timeout to work around occasional hangs (GoogleCloudPlatformGH-483) * Fixed lock pickling issues (GoogleCloudPlatformGH-471) * Added a “-v” flag (GoogleCloudPlatformGH-463)
Release 0.21.0. Changes in this release: * New features: * Add support for OpenStack affinity and anti-affinity groups. (GoogleCloudPlatformGH-440, thanks to @kivio) * Add large object scenario for CLI tests in object_storage_service. (GoogleCloudPlatformGH-445) * Add support for pre-emptible GCE VMs. (GoogleCloudPlatformGH-415) * Enhancements: * Parallelize cassandra_ycsb and hbase_ycsb benchmark setups. (GoogleCloudPlatformGH-435) * Add more config options for aerospike benchmark. (GoogleCloudPlatformGH-450) * Bugfixes and maintenance updates: * Refactor OpenStack network IP management. (GoogleCloudPlatformGH-438, thanks to @kivio) * Fix thread lock pickling bug. (GoogleCloudPlatformGH-425, thanks to @kivio) * Update jars used in cloud_bigtable_ycsb benchmark. (GoogleCloudPlatformGH-444) * Update YCSB to v0.3.0. (GoogleCloudPlatformGH-428) * Fix object_storage_service bug introduced by Azure breaking change. (GoogleCloudPlatformGH-446) * Fix object storage CLI output format. (GoogleCloudPlatformGH-449)
Release v0.20.0 * Enhancements: * Specify project to gcloud alpha bigtable clusters list (GoogleCloudPlatformGH-433) * More Samples in FIO (GoogleCloudPlatformGH-416) * Bugfixes and maintenance updates: * pkb label problem in OpenStack driver fixes (GoogleCloudPlatformGH-410) * On exception, only cleanup if run stage is all/cleanup (GoogleCloudPlatformGH-412) * Fix issue with using a flag before flags are parsed (GoogleCloudPlatformGH-413) * Umount disk when running fio against raw device. (GoogleCloudPlatformGH-417) * Clarify warnings from ycsb._CombineResults. (GoogleCloudPlatformGH-432)
v0.19.0
* New features:
* New mysql_service benchmark. This benchmarks a cloud's managed MySQL
offering using sysbench. (GoogleCloudPlatformGH-387)
* Enhancements:
* Added option to disable iptables if your image requires it (GoogleCloudPlatformGH-361)
* mongodb_ycsb now installs client and server dependencies in parallel,
speeding up the end to end run time for the benchmark. (GoogleCloudPlatformGH-402)
* The netperf and iperf benchmarks now only add firewall rules if they are
running over external ips. (GoogleCloudPlatformGH-382)
* Bugfixes and maintenance updates:
* The iperf package will now check the 'redhat-release' version and install
directly from an RPM (this enables iperf to be run on Scientific Linux 6.x).
(GoogleCloudPlatformGH-392, thanks to @vukasin92)
* Fix bug where VM temporary directory wasn't created before use on RHEL based
static VMs. (GoogleCloudPlatformGH-389, thanks to @vukasin92)
* netperf package url changed since version 2.6.0 is now in archive/ (GoogleCloudPlatformGH-390)
* Fixed DigitalOcean package installation error (GoogleCloudPlatformGH-396)
* The object_storage_service benchmark no longer copies gcloud logs as part of copying
the gcloud configuration to the VM. (GoogleCloudPlatformGH-383)
* Correctly cleanup network resources when run stages are used. (GoogleCloudPlatformGH-386)
* Added timeout to apt-get update command because it will occasionally hang.
(GoogleCloudPlatformGH-391)
* Update copy_throughput benchmark so it works with
ContainerizedVirtualMachines. (GoogleCloudPlatformGH-408)
* Install python inside ContainerizedVirtualMachines so that
RobustRemoteCommand works on them. (GoogleCloudPlatformGH-404)
* Benchmark-specific changes:
* speccpu2006 will no longer report results if the run was incomplete. This
behavior can be modified with a flag. (GoogleCloudPlatformGH-397)
* The mongodb benchmark has been completely removed since mongodb_ycsb
replaced it with greater functionality. (GoogleCloudPlatformGH-403)
* The fio benchmark now has more latency percentiles included in sample
metadata. (GoogleCloudPlatformGH-399)
* Cassandra version bumped up to 2.0.16 since 2.0.0 has known issues (GoogleCloudPlatformGH-393)
Release 0.18.0. (See also GoogleCloudPlatform#369 which includes this change log with clickable GH-* links.) * New features: * Support OpenStack as cloud provider (GoogleCloudPlatformGH-305, GoogleCloudPlatformGH-353, thanks @kivio and @mateusz-blaszkowski) * Support Rackspace as cloud provider (GoogleCloudPlatformGH-336, thanks @meteorfox and @jrperritt) * Add support for ContainerizedVM using docker exec (GoogleCloudPlatformGH-333, thanks @gablg1) * Windows guest VM support on Static VM (GoogleCloudPlatformGH-350), Azure (GoogleCloudPlatformGH-349, GoogleCloudPlatformGH-374), AWS (GoogleCloudPlatformGH-347), and GCE (GoogleCloudPlatformGH-338) * Add NTttcp Windows networking benchmark (GoogleCloudPlatformGH-348) * Enhancements: * Support using proxies in VMs (GoogleCloudPlatformGH-339, GoogleCloudPlatformGH-337, thanks @kivio) * Enable optional migration on GCE (GoogleCloudPlatformGH-343) * Execute long running commands via a remote agent (GoogleCloudPlatformGH-310) * Add resource creation/deletion times to logs (GoogleCloudPlatformGH-316) * Bugfixes and maintenance updates: * Update PKB to work with Azure version 0.9.3 (GoogleCloudPlatformGH-312) * Fix AWS CLI usage on Windows host (GoogleCloudPlatformGH-313) * Auto-fetch AMI IDs for AWS images (GoogleCloudPlatformGH-364) * Fix publisher missing info for default image and machine type (GoogleCloudPlatformGH-357) * Fix 'no attribute pkb_thread_log_context' error for sub-thread logs (GoogleCloudPlatformGH-322) * Benchmark-specific changes: * aerospike: config/flag handling bugfixes (GoogleCloudPlatformGH-367, GoogleCloudPlatformGH-360, GoogleCloudPlatformGH-354) * cassandra_ycsb: move num_vms prerequisite check * fio: add latency percentiles for results (GoogleCloudPlatformGH-344) * hadoop_terasort: Fix bad SSH option (GoogleCloudPlatformGH-328) * iperf: add lower bounds to arguments (GoogleCloudPlatformGH-314) * iperf: add timeout to parallel benchmark runs to handle iperf hangs (GoogleCloudPlatformGH-375) * netperf: Support confidence intervals, increase test length, report stddev (GoogleCloudPlatformGH-317, GoogleCloudPlatformGH-306) * ycsb: Drop unaggregatable results from samples (GoogleCloudPlatformGH-324) * Development and testing: * **Breaking Change** Automated testing now uses `tox` (GoogleCloudPlatformGH-330) * Refactored hook scripts, including new opt-in pre-push hook (GoogleCloudPlatformGH-363) * Use travis for CI testing (GoogleCloudPlatformGH-340) * Speed up tests using timeouts (GoogleCloudPlatformGH-299) * Internals: * Move defaults from benchmark_spec to VM classes, move network instantiation out of benchmark spec (GoogleCloudPlatformGH-342) * Add event hook support (GoogleCloudPlatformGH-315) * Refactor VM classes (GoogleCloudPlatformGH-321)
Release 0.17.0. Changes in this release: * Add initial support for DigitalOcean as a cloud provider (GoogleCloudPlatformGH-291). * Add SciMark2 Benchmark (GoogleCloudPlatformGH-271, thanks @zlim!). * New NoSQL benchmarks: `cloud_bigtable_ycsb` (GoogleCloudPlatformGH-283), `mongodb_ycsb` (GoogleCloudPlatformGH-279), `cassandra_ycsb` (GoogleCloudPlatformGH-278). * Allow PerfKitBenchmarker to run on a Windows controller (GoogleCloudPlatformGH-274). * Add a `--dstat` flag, to collect performance metrics during benchmark runs (GoogleCloudPlatformGH-282). * Add support for parallel iperf clients to the iperf benchmark (GoogleCloudPlatformGH-262). * Add scratch disk information to sample metadata (GoogleCloudPlatformGH-277). * Add a 5 minute timeout to local commands (GoogleCloudPlatformGH-289, GoogleCloudPlatformGH-293). * Do not use FLAG values in generating benchmark documentation (GoogleCloudPlatformGH-280). * Bump HBase to v1.0.1.1 (GoogleCloudPlatformGH-281). * Fix an issue preventing resources from being cleaned up (GoogleCloudPlatformGH-276).
PreviousNext