8000 Start crate smashing by tcharding · Pull Request #751 · rust-bitcoin/rust-bitcoin · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Start crate smashing #751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 3 additions & 60 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,60 +1,3 @@
[package]
name = "bitcoin"
version = "0.27.0"
authors = ["Andrew Poelstra <apoelstra@wpsoftware.net>"]
license = "CC0-1.0"
homepage = "https://github.com/rust-bitcoin/rust-bitcoin/"
repository = "https://github.com/rust-bitcoin/rust-bitcoin/"
documentation = "https://docs.rs/bitcoin/"
description = "General purpose library for using and interoperating with Bitcoin and other cryptocurrencies."
keywords = [ "crypto", "bitcoin" ]
readme = "README.md"
exclude = ["./test_data"]

# Please don't forget to add relevant features to docs.rs below
[features]
default = [ "std", "secp-recovery" ]
base64 = [ "base64-compat" ]
unstable = []
rand = ["secp256k1/rand-std"]
use-serde = ["serde", "bitcoin_hashes/serde", "secp256k1/serde"]
secp-lowmemory = ["secp256k1/lowmemory"]
secp-recovery = ["secp256k1/recovery"]

# At least one of std, no-std must be enabled.
#
# The no-std feature doesn't disable std - you need to turn off the std feature for that by disabling default.
# Instead no-std enables additional features required for this crate to be usable without std.
# As a result, both can be enabled without conflict.
std = ["secp256k1/std", "bitcoin_hashes/std", "bech32/std"]
no-std = ["hashbrown", "core2/alloc", "bitcoin_hashes/alloc", "secp256k1/alloc"]

[package.metadata.docs.rs]
features = [ "std", "secp-recovery", "base64", "rand", "use-serde", "bitcoinconsensus" ]
rustdoc-args = ["--cfg", "docsrs"]

[dependencies]
bech32 = { version = "0.8.1", default-features = false }
bitcoin_hashes = { version = "0.10.0", default-features = false }
secp256k1 = { version = "0.21.2", default-features = false }
core2 = { version = "0.3.0", optional = true, default-features = false }

base64-compat = { version = "1.0.0", optional = true }
bitcoinconsensus = { version = "0.19.0-3", optional = true }
serde = { version = "1", features = [ "derive" ], optional = true }
hashbrown = { version = "0.8", optional = true }

[dev-dependencies]
serde_json = "<1.0.45"
serde_test = "1"
secp256k1 = { version = "0.21.2", features = [ "recovery", "rand-std" ] }
bincode = "1.3.1"
# We need to pin ryu (transitive dep from serde_json) to stay compatible with Rust 1.22.0
ryu = "<1.0.5"

[[example]]
name = "bip32"

[[example]]
name = "handshake"
required-features = ["std"]
[workspace]
members = ["bitcoin", "bitcoin-str"]
exclude = ["embedded", "fuzz"]
Comment on lines +1 to +3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can reduce the diff here by not moving the main crate into a subdirectory.

Regular crate manifests can also have a [workspace] table, meaning you can introduce bitcoin-str and move all the stuff in without touching anything in bitcoin.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh cool, I did not know that. Cheers man!

41 changes: 41 additions & 0 deletions bitcoin-str/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
[package]
name = "bitcoin-str"
version = "0.27.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
version = "0.27.0"
version = "0.1.0"

authors = ["Andrew Poelstra <apoelstra@wpsoftware.net>"]
license = "CC0-1.0"
homepage = "https://github.com/rust-bitcoin/rust-bitcoin/"
repository = "https://github.com/rust-bitcoin/rust-bitcoin/"
documentation = "https://docs.rs/bitcoin/"
description = "The string encodings used by rust-bitcoin (base58, base58check, bech32, bech32m, base64, hex)."
keywords = [ "crypto", "bitcoin" ]
readme = "https://github.com/rust-bitcoin/rust-bitcoin/README.md"

# Please don't forget to add relevant features to docs.rs below
[features]
default = ["std"]
unstable = []
base64 = ["base64-compat"]
use-serde = ["serde", "bitcoin_hashes/serde"]

# At least one of std, no-std must be enabled.
#
# The no-std feature doesn't disable std - you need to turn off the std feature for that by disabling default.
# Instead no-std enables additional features required for this crate to be usable without std.
# As a result, both can be enabled without conflict.
std = ["bitcoin_hashes/std", "bech32/std"]
no-std = ["core2/alloc", "bitcoin_hashes/alloc"]

[package.metadata.docs.rs]
features = [ "std", "secp-recovery", "base64", "rand", "use-serde", "bitcoinconsensus" ]
rustdoc-args = ["--cfg", "docsrs"]

[dependencies]
bech32 = { version = "0.8.1", default-features = false }
bitcoin_hashes = { version = "0.10.0", default-features = false }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the hashes as well or are we just taking hex from this? Would be nice to move hex into bitcoin-str and have bitcoin-hashes depend on bitcoin-str. AFAIK, hex only lives in bitcoin-hashes because bitcoin-hashes was already a leaf dependency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

concept ACK that dependency inversion.

Yes, the hex stuff was put into bitcoin-hashes purely because it was the smallest crate in the ecosystem. (I might've put hex into bech32 if it'd been around at the time :P.)

Copy link
Member Author
@tcharding tcharding Jan 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are three functions in the base58 module that depend on hashing for checksum'ing, e.g.

/// Obtain a string with the base58check encoding of a slice
/// (Tack the first 4 256-digits of the object's Bitcoin hash onto the end.)
pub fn check_encode_slice(data: &[u8]) -> String {
    let checksum = sha256d::Hash::hash(data);
    encode_iter(
        data.iter()
            .cloned()
            .chain(checksum[0..4].iter().cloned())
    )
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping to convey that with the comment in bitcoin-str/src/lib.rs

pub mod hex {
    //! Re-exports hex de/serialization code from `bitcoin_hashes`.
    //!
    // There is a chicken and the egg problem with putting _all_ the en/decoding code in a single crate.
    // Hashing cannot be separated totally from en/decoding because base58 -> hashing -> hex.

Or is there a way around this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we don't use the hex crate?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading your argument I no longer feel comfortable with accepting option 3.

I believe we should really revive it. But could the crate be without "bitcoin" in its name or docs? I'd like it being neutral general purpose 1.36-compatible crate. hex-compat similarly to base64-compat sounds sensible.

Copy link
Collaborator
@dr-orlovsky dr-orlovsky Jan 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is already amplify_num crate I made for this purpose, when work on bitcoin_hex has stalled. I will be ok with moving it to this org and even renaming it. Additionally to hex it has big and small integers (u1-u7, u256 taken from rust-bitcoin ecosystem and u512-u1024). u256 as multiple improvements over type in rust-bitcoin, providing arithmetics, overflow wraping/checkings, octal/hex etc formating – but the code base is still quite small.

Currently it is a part of the repo managed by me and @Kixunil: https://github.com/LNP-BP/rust-amplify

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unconvinced that "changes will happen" to hex encoding code which require synchronized updates, but concept ACK having a separate hex-compat crate.

Agree that it shouldn't have bitcoin anywhere in the name.

I'm also OK in concept with talking to the hex maintainer but I have no interest in doing this myself.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unconvinced that "changes will happen" to hex encoding code which require synchronized updates, but concept ACK having a separate hex-compat crate.

That was my thinking as well. Very likely, the hex-encoding needs within bitcoin-hashes are not going to change any time soon and hence we might be able to reduce it down to 2-3 private functions that have exactly the signatures we need because the don't need to account for any other usage.

Contrary to that, a public hex module within bitcoin-str will likely need to offer much more flexible APIs, see more usage and thus require more maintenance.

That was the thinking when I proposed option 3 above. I am not fussed about extracting a dedicated crate though.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I've mentioned there that I wish for MSRV to be no newer than 1.41.1 and noticed they also have 1.0 milestone which is very nice. I can see that the code being short and unlikely to change can be a good argument for duplication being fine but still prefer a separate crate. It could be a fork of hex with changes to make it compilable on 1.41.1.


serde = { version = "1", features = [ "derive" ], optional = true } # Typically enabled with `use-serde`.
base64-compat = { version = "1.0.0", optional = true } # Typically enabled with `base64`.
core2 = { version = "0.3.0", default-features = false, optional = true } # Typically enabled with `no-std`.

[dev-dependencies]

17 changes: 1 addition & 16 deletions src/util/base58.rs → bitcoin-str/src/base58.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,7 @@ use prelude::*;
use core::{fmt, str, iter, slice};

use hashes::{sha256d, Hash};
use secp256k1;

use util::{endian, key};
use endian;

/// An error that might occur during base58 decoding
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Clone)]
Expand All @@ -44,8 +42,6 @@ pub enum Error {
InvalidAddressVersion(u8),
/// Checked data was less than 4 bytes
TooShort(usize),
/// Secp256k1 error while parsing a secret key
Secp256k1(secp256k1::Error),
}

impl fmt::Display for Error {
Expand All @@ -57,7 +53,6 @@ impl fmt::Display for Error {
Error::InvalidAddressVersion(ref v) => write!(f, "address version {} is invalid for this base58 type", v),
Error::InvalidExtendedKeyVersion(ref v) => write!(f, "extended key version {:#04x?} is invalid for this base58 type", v),
Error::TooShort(_) => write!(f, "base58ck data not even long enough for a checksum"),
Error::Secp256k1(ref e) => fmt::Display::fmt(&e, f),
}
}
}
Expand Down Expand Up @@ -249,16 +244,6 @@ pub fn check_encode_slice_to_fmt(fmt: &mut fmt::Formatter, data: &[u8]) -> fmt::
format_iter(fmt, iter)
}

#[doc(hidden)]
impl From<key::Error> for Error {
fn from(e: key::Error) -> Self {
match e {
key::Error::Secp256k1(e) => Error::Secp256k1(e),
key::Error::Base58(e) => e,
}
}
}

#[cfg(test)]
mod tests {
use super::*;
Expand Down
2 changes: 2 additions & 0 deletions src/util/endian.rs → bitcoin-str/src/endian.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#![allow(missing_docs)] // TODO: Work out how to add valid docs during macro implementation of functions.

macro_rules! define_slice_to_be {
($name: ident, $type: ty) => {
#[inline]
Expand Down
91 changes: 91 additions & 0 deletions bitcoin-str/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
// Rust Bitcoin Library
// Written in 2014 by
// Andrew Poelstra <apoelstra@wpsoftware.net>
//
// To the extent possible un F438 der law, the author(s) have dedicated all
// copyright and related and neighboring rights to this software to
// the public domain worldwide. This software is distributed without
// any warranty.
//
// You should have received a copy of the CC0 Public Domain Dedication
// along with this software.
// If not, see <http://creativecommons.org/publicdomain/zero/1.0/>.
//

//! # Rust Bitcoin String Encoding/Decoding.
//!
//! This library provides encoding and decoding into/from various string formats used by Bitcoin.
//!
//! - base58
//! - bech32
//! - bech32m
//! - hex
//! - base64 (optional)
//!

#![cfg_attr(all(not(feature = "std"), not(test)), no_std)]

// Experimental features we need
#![cfg_attr(all(test, feature = "unstable"), feature(test))]

#![cfg_attr(docsrs, feature(doc_cfg))]

// Coding conventions
#![forbid(unsafe_code)]
#![deny(non_upper_case_globals)]
#![deny(non_camel_case_types)]
#![deny(non_snake_case)]
#![deny(unused_mut)]
#![deny(dead_code)]
#![deny(unused_imports)]
#![deny(missing_docs)]
#![deny(unused_must_use)]
#![deny(broken_intra_doc_links)]

#[cfg(not(any(feature = "std", feature = "no-std")))]
compile_error!("at least one of the `std` or `no-std` features must be enabled");

// Disable 16-bit support at least for now as we can't guarantee it yet.
#[cfg(target_pointer_width = "16")]
compile_error!("rust-bitcoin currently only supports architectures with pointers wider
than 16 bits, let us know if you want 16-bit support. Note that we do
NOT guarantee that we will implement it!");

#[cfg(feature = "no-std")]
#[macro_use]
extern crate alloc;
#[cfg(feature = "no-std")]
extern crate core2;

#[cfg(any(feature = "std", test))]
extern crate core; // for Rust 1.29 and no-std tests

// Re-exported dependencies.
pub extern crate bech32;
pub extern crate bitcoin_hashes as hashes;

#[cfg(feature = "base64")]
#[cfg_attr(docsrs, doc(cfg(feature = "base64")))]
pub extern crate base64;

pub mod base58;
pub mod endian;

pub mod hex {
//! Re-exports hex de/serialization code from `bitcoin_hashes`.
//!
// There is a chicken and the egg problem with putting _all_ the en/decoding code in a single crate.
// Hashing cannot be separated totally from en/decoding because base58 -> hashing -> hex.

pub use hashes::hex::{Error, ToHex, FromHex, HexIterator, format_hex, format_hex_reverse};
// TODO: This doesn't work with Rust 1.29 (same problem trying to use macros from hashes in ../bitcoin)
pub use hashes::hex_fmt_impl;
}

mod prelude {
#[cfg(all(not(feature = "std"), not(test)))]
pub use alloc::{string::{String, ToString}, vec::Vec, boxed::Box, borrow::{Cow, ToOwned}, slice, rc, sync};

#[cfg(any(feature = "std", test))]
pub use std::{string::{String, ToString}, vec::Vec, boxed::Box, borrow::{Cow, ToOwned}, slice, rc, sync};
}
59 changes: 59 additions & 0 deletions bitcoin/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
[package]
name = "bitcoin"
version = "0.27.0"
authors = ["Andrew Poelstra <apoelstra@wpsoftware.net>"]
license = "CC0-1.0"
homepage = "https://github.com/rust-bitcoin/rust-bitcoin/"
repository = "https://github.com/rust-bitcoin/rust-bitcoin/"
documentation = "https://docs.rs/bitcoin/"
description = "General purpose library for using and interoperating with Bitcoin and other cryptocurrencies."
keywords = [ "crypto", "bitcoin" ]
readme = "https://github.com/rust-bitcoin/rust-bitcoin/README.md"
exclude = ["./test_data"]

# Please don't forget to add relevant features to docs.rs below
[features]
default = [ "std", "secp-recovery" ]
base64 = [ "bitcoin-str/base64" ]
unstable = []
rand = ["secp256k1/rand-std"]
use-serde = ["serde", "secp256k1/serde"]
secp-lowmemory = ["secp256k1/lowmemory"]
secp-recovery = ["secp256k1/recovery"]

# At least one of std, no-std must be enabled.
#
# The no-std feature doesn't disable std - you need to turn off the std feature for that by disabling default.
# Instead no-std enables additional features required for this crate to be usable without std.
# As a result, both can be enabled without conflict.
std = ["bitcoin-str/std", "secp256k1/std"]
no-std = ["bitcoin-str/no-std", "hashbrown", "core2/alloc", "secp256k1/alloc"]

[package.metadata.docs.rs]
features = [ "std", "secp-recovery", "base64", "rand", "use-serde", "bitcoinconsensus" ]
rustdoc-args = ["--cfg", "docsrs"]

[dependencies]
bitcoin-str = { path = "../bitcoin-str", default-features = false }

secp256k1 = { version = "0.21.2", default-features = false }
core2 = { version = "0.3.0", optional = true, default-features = false }

bitcoinconsensus = { version = "0.19.0-3", optional = true }
serde = { version = "1", features = [ "derive" ], optional = true }
hashbrown = { version = "0.8", optional = true }

[dev-dependencies]
serde_json = "<1.0.45"
serde_test = "1"
secp256k1 = { version = "0.21.2", features = [ "recovery", "rand-std" ] }
bincode = "1.3.1"
# We need to pin ryu (transitive dep from serde_json) to stay compatible with Rust 1.22.0
ryu = "<1.0.5"

[[example]]
name = "bip32"

[[example]]
name = "handshake"
required-features = ["std"]
File renamed without changes.
File renamed without changes.
5 changes: 3 additions & 2 deletions src/blockdata/block.rs → bitcoin/src/blockdata/block.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ use blockdata::transaction::Transaction;
use blockdata::constants::{max_target, WITNESS_SCALE_FACTOR};
use blockdata::script;
use VarInt;
use endian;

/// A block header, which contains all the block's information except
/// the actual transactions
Expand Down Expand Up @@ -138,7 +139,7 @@ impl BlockHeader {
}
let block_hash = self.block_hash();
let mut ret = [0u64; 4];
util::endian::bytes_to_u64_slice_le(block_hash.as_inner(), &mut ret);
endian::bytes_to_u64_slice_le(block_hash.as_inner(), &mut ret);
let hash = &Uint256(ret);
if hash <= target { Ok(block_hash) } else { Err(BlockBadProofOfWork) }
}
Expand Down Expand Up @@ -293,7 +294,7 @@ impl Block {
// Expand the push to exactly 8 bytes (LE).
let mut full = [0; 8];
full[0..b.len()].copy_from_slice(b);
Ok(util::endian::slice_to_u64_le(&full))
Ok(endian::slice_to_u64_le(&full))
}
script::Instruction::PushBytes(b) if b.len() > 8 => {
Err(Bip34Error::UnexpectedPush(b.to_vec()))
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ use core::{fmt, str, default::Default};
use hashes::{self, Hash, sha256d};
use hashes::hex::FromHex;

use util::endian;
use endian;
use blockdata::constants::WITNESS_SCALE_FACTOR;
#[cfg(feature="bitcoinconsensus")] use blockdata::script;
use blockdata::script::Script;
Expand Down
File renamed without changes.
4 changes: 2 additions & 2 deletions src/consensus/encode.rs → bitcoin/src/consensus/encode.rs
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ use hash_types::{BlockHash, FilterHash, TxMerkleNode, FilterHeader};

use io::{self, Cursor, Read};

use util::endian;
use endian;
use util::psbt;
use util::taproot::TapLeafHash;
use hashes::hex::ToHex;
Expand Down Expand Up @@ -789,7 +789,7 @@ mod tests {
use super::{deserialize, serialize, Error, CheckedData, VarInt};
use super::{Transaction, BlockHash, FilterHash, TxMerkleNode, TxOut, TxIn};
use consensus::{Encodable, deserialize_partial, Decodable};
use util::endian::{u64_to_array_le, u32_to_array_le, u16_to_array_le};
use endian::{u64_to_array_le, u32_to_array_le, u16_to_array_le};
use secp256k1::rand::{thread_rng, Rng};
#[cfg(feature = "std")]
use network::{Address, message_blockdata::Inventory};
Expand Down
File renamed without changes.
File renamed without changes.
4 changes: 3 additions & 1 deletion src/hash_types.rs → bitcoin/src/hash_types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@
//! hash).
//!

use hashes::{Hash, sha256, sha256d, hash160};
use hashes::{Hash, sha256, sha256d, hash160, hash_newtype};
// TODO: These 'hidden' macros should use fully qualified paths in bitcoin_hashes.
use hashes::{index_impl, hex_fmt_impl, serde_impl, borrow_slice_impl};

macro_rules! impl_hashencode {
($hashtype:ident) => {
Expand Down
File renamed without changes.
Loading
0