- Boost-users - lists.preview.boost.org

Iostreams with two filters throws exception
by NuSkooler 12 Aug '13

12 Aug '13

I've been attempting to use Boost.Iostreams with two filters to compress + encrypt ("out") and decrypt + uncompress ("in"). This works for some files, while others throw an exception during the "in" phase. If I remove either of the filters for out/in (e.g. remove the compress or the encrypt) everything works fine -- the issue only occurs with two filters chained. >From what I understand in the documentation, this should be supported, but I must be missing something. Below is the relevant portions of my code: Input & output cipher filters (note: I'm using Boost.Crypto, and boost::crypto::rc4_cipher specifically) template <typename StreamCipherT> class StreamCipherFilterBase { public: typedef char char_type; StreamCipherFilterBase(const std::string& key) { m_cipher.set_key(key.data(), static_cast<unsigned int>(key.length())); } protected: char m_buffer[1024 * 2]; // 2k internal buffer size StreamCipherT m_cipher; }; template <typename StreamCipherT> class StreamCipherEncryptFilter : public StreamCipherFilterBase<StreamCipherT> { public: typedef boost::iostreams::multichar_output_filter_tag category; StreamCipherEncryptFilter(const std::string& key) : StreamCipherFilterBase<StreamCipherT>(key) { } template<typename Sink> std::streamsize write(Sink& snk, const char* s, std::streamsize n) { std::streamsize written = 0; std::streamsize remain = n; std::streamsize eat; while(remain) { eat = remain < sizeof(StreamCipherFilterBase<StreamCipherT>::m_buffer) ? remain : sizeof(StreamCipherFilterBase<StreamCipherT>::m_buffer); StreamCipherFilterBase<StreamCipherT>::m_cipher.encrypt( s, StreamCipherFilterBase<StreamCipherT>::m_buffer, eat); written += boost::iostreams::write(snk, StreamCipherFilterBase<StreamCipherT>::m_buffer, eat); remain -= eat; s += eat; } return written; } }; template <typename StreamCipherT> class StreamCipherDecryptFilter : public StreamCipherFilterBase<StreamCipherT> { public: typedef boost::iostreams::multichar_input_filter_tag category; StreamCipherDecryptFilter(const std::string& key) : StreamCipherFilterBase<StreamCipherT>(key) { } template<typename Source> std::streamsize read(Source& src, char* s, std::streamsize n) { const std::streamsize read = boost::iostreams::read(src, s, n); if(EOF == read) { return EOF; } StreamCipherFilterBase<StreamCipherT>::m_cipher.decrypt(s, s, read); return read; } }; And compress + encrypt ("out") & decrypt + uncompress ("in"): (error checking removed for this post) // // Write out data compressing and encrypting as we go // boost::filesystem::ifstream inf(inFile.Get(), std::ios_base::binary | std::ios_base::in); boost::filesystem::ofstream outf(outFile.Get(), std::ios_base::binary | std::ios_base::out); StreamCipherEncryptFilter<CipherT> outCipherFilter(key); boost::iostreams::filtering_ostreambuf out; out.push(boost::iostreams::zlib_compressor()); // compress out.push(outCipherFilter); // then encrypt out.push(outf); // write to file boost::iostreams::copy(inf, out); // // Write out decrypted and uncompressed data: // boost::filesystem::ifstream inf(inFile.Get(), std::ios_base::binary | std::ios_base::in); boost::filesystem::ofstream outf(outFile.Get(), std::ios_base::binary | std::ios_base::out); StreamCipherDecryptFilter<CipherT> inCipherFilter(key); boost::iostreams::filtering_istreambuf in; in.push(boost::iostreams::zlib_decompressor()); in.push(inCipherFilter); in.push(inf); boost::iostreams::copy(in, outf); // <--- exception is thrown here for many files; some succeed with input md5 = output md5. Can anyone help? Bryan Ashby

2 2

Re: [Boost-users] [Interprocess] Page default
by Kelly, Dan 12 Aug '13

12 Aug '13

To elaborate on Ion's remarks, ALL virtual memory in user space in windows is backed by the page file, and ALL file I/O is memory mapped, the only difference being virtual memory is backed by the page file rather than a specific named file in the file system. Your inter process memory is just as likely to be in physical memory as any other data. So most likely this is not the cause of your performance issue, unless memory is so tight that your entire process working set is continually thrashing. Dan -----Original Message----- From: Ion Gaztañaga [igaztanaga(a)gmail.com<mailto:igaztanaga@gmail.com>] Sent: Wednesday, July 24, 2013 06:48 PM Eastern Standard Time To: boost-users(a)lists.boost.org Subject: Re: [Boost-users] [Interprocess] Page default El 18/07/2013 14:45, Oodini escribió: > Hello, > > I use Boost.Interprocess on Windows 2003 Server. > > I am experiencing performances troubles. I am trying to understand > what happens, and arrived on Microsoft web pages. > > I seems that on Windows, shared memory is implemented only through > mapped files, and that only a portion of the shared memory is > physically in the RAM (this limited number of pages is called > "working set"). > > The Windows API provides some functions for this working set : > http://msdn.microsoft.com/en-us/library/windows/desktop/cc441804%28v=vs.85%… > > I would have liked Interprocess wraps the functions > GetProcessMemoryInfo() and SetProcessWorkingSetSize(), so that the > full shared memory is put in RAM. Inteprocess can't do that because it's a portable library and POSIX systems don't know what a working set is. You can perfectly call those functions yourself as you don't need any internal Interprocess data to call them. Although shared memory is never guaranteed to be in RAM, you don't any any guarantee with your "process memory" (if memory pressure is detected the OS can swap you heap and/or executable pages to the swap file to free some RAM for other processes). Best, Ion _______________________________________________ Boost-users mailing list Boost-users(a)lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users ________________________________ IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

3 3

[zlib] Compression intended use
by Michael Powell 11 Aug '13

11 Aug '13

Hello, This is probably out of scope with boost's intended usage pattern, but I'll ask anyhow. I've got a package of output files that I want to bundle, tarball, gzip, or however I can manage to compress them. I see there is the zlib usage, but methinks that is more for in-memory compression. Not sure exactly how that translates to folders-/files-oriented compression I am thinking of. Thank you... Regards, Michael Powell

3 2

Re: [Boost-users] Iostreams with two filters throws exception (Ken Appleby)
by NuSkooler 10 Aug '13

10 Aug '13

Below is a little app that illustrates the issue (Can also be found here: http://pastebin.com/k8jRN64R) I've removed most error handling/etc. for illustration purposes. The program expects a "input.txt" to be present in the same directory it's run from. Note again that this works *most* of the time, but depending on the input file, will crash. Also I forgot to mention before that I'm using Clang on OS X 8.4 and Boost v1.53. Any help is greatly appreciated! // Boost.Crypto #include <boost/crypto/rc4_cipher.hpp> // Boost #include <boost/filesystem/fstream.hpp> #include <boost/iostreams/filter/zlib.hpp> #include <boost/iostreams/copy.hpp> #include <boost/iostreams/filtering_stream.hpp> template <typename StreamCipherT> class StreamCipherFilterBase { public: typedef char char_type; StreamCipherFilterBase(const std::string& key) { m_cipher.set_key(key.data(), static_cast<unsigned int>(key.length())); } protected: char m_buffer[1024 * 2]; // 2k internal buffer size StreamCipherT m_cipher; }; template <typename StreamCipherT> class StreamCipherEncryptFilter : public StreamCipherFilterBase<StreamCipherT> { public: typedef boost::iostreams::multichar_output_filter_tag category; StreamCipherEncryptFilter(const std::string& key) : StreamCipherFilterBase<StreamCipherT>(key) { } template<typename Sink> std::streamsize write(Sink& snk, const char* s, std::streamsize n) { std::streamsize written = 0; std::streamsize remain = n; std::streamsize eat; while(remain) { eat = remain < sizeof(StreamCipherFilterBase<StreamCipherT>::m_buffer) ? remain : sizeof(StreamCipherFilterBase<StreamCipherT>::m_buffer); StreamCipherFilterBase<StreamCipherT>::m_cipher.encrypt( s, StreamCipherFilterBase<StreamCipherT>::m_buffer, eat); written += boost::iostreams::write(snk, StreamCipherFilterBase<StreamCipherT>::m_buffer, eat); remain -= eat; s += eat; } return written; } }; template <typename StreamCipherT> class StreamCipherDecryptFilter : public StreamCipherFilterBase<StreamCipherT> { public: typedef boost::iostreams::multichar_input_filter_tag category; StreamCipherDecryptFilter(const std::string& key) : StreamCipherFilterBase<StreamCipherT>(key) { } template<typename Source> std::streamsize read(Source& src, char* s, std::streamsize n) { const std::streamsize read = boost::iostreams::read(src, s, n); if(EOF == read) { return EOF; } StreamCipherFilterBase<StreamCipherT>::m_cipher.decrypt(s, s, read); return read; } }; const std::string KEY = "Kitteh"; typedef boost::crypto::rc4_cipher CipherT; void Encrypt( const std::string& in, const std::string& out) { // error handling omitted boost::filesystem::ifstream inf(in, std::ios_base::binary | std::ios_base::in); boost::filesystem::ofstream outf(out, std::ios_base::binary | std::ios_base::out); StreamCipherEncryptFilter<CipherT> outCipherFilter(KEY); boost::iostreams::filtering_ostreambuf outs; outs.push(boost::iostreams::zlib_compressor()); // compress outs.push(outCipherFilter); // then encrypt outs.push(outf); // write to file boost::iostreams::copy(inf, outs); } void Decrypt( const std::string& in, const std::string& out) { // error handling omitted boost::filesystem::ifstream inf(in, std::ios_base::binary | std::ios_base::in); boost::filesystem::ofstream outf(out, std::ios_base::binary | std::ios_base::out); StreamCipherDecryptFilter<CipherT> inCipherFilter(KEY); boost::iostreams::filtering_istreambuf ins; ins.push(boost::iostreams::zlib_decompressor()); ins.push(inCipherFilter); ins.push(inf); boost::iostreams::copy(ins, outf); // <-- exception thrown here with some files } int main() { // compress+encrypt input.txt -> output.encrypt Encrypt("input.txt", "output.encrypt"); // decrypt+decompress output.encrypt -> input.read.txt Decrypt("output.encrypt", "input.read.txt"); } ------------------------------ > > Message: 5 > Date: Thu, 08 Aug 2013 17:52:50 +0100 > From: Ken Appleby <ken.appleby(a)btopenworld.com> > To: boost-users(a)lists.boost.org > Subject: Re: [Boost-users] Iostreams with two filters throws exception > Message-ID: <5203CCE2.4060200(a)btopenworld.com> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > > You are a lot more likely to get a response if you post a complete > example. Just a single file with a main() would make it a lot easier to > help you. You're not far from that with the code you posted. > >

2 1

Boost.Variant broke in 1.54?
by gast128 09 Aug '13

09 Aug '13

Dear all, I get a compilation error with Boost.Variant (which compiles fine on previous releases) with vstudio 2010 when a function returns a const Boost.Variant, e.g.: typedef boost::variant<int, double> Variant; const Variant GetVar() { return Variant(); } void TestVariantDnc() { const Variant var = GetVar(); } This gives error C2666 ('boost::variant<T0_,T1>::convert_construct' : 3 overloads have similar conversions'). Shall a file a ticket or do I something wrong?

4 8

Releasing Boost Dependency Analyzer 1.1
by Jens Weller 09 Aug '13

09 Aug '13

Hello boost community, a short notice that I just have released the version 1.1 of my dependency analyzing tool for boost. For now I have only managed to get a proper Windows Version, Linux Version will follow later. New Features: Configuration - the tool remembers the path to bcp now - select the source for libraries - maintainers.txt - libraries.xml - 2 filter options - filter libraries by name - filter edges by type, for now its header, source/lib, test and documentation. Export: - Export CSV File which has all libraries and the number of dependencies (sorted) libraries.xml - this is now a optional source for the library names Some feedback: libraries.xml as a lib name source is nice, but not quite perfect. I've seen different results from maintainers.txt and libraries.xml. As maintainers.txt also libraries.xml contains duplicates, some of its library names don't fit the expectation of bcp, so I applied some fixes for this in my program. The Dependency filter is yet not perfect, especially in detection. More details about this version and the download for it at my blog: http://www.meetingcpp.com/index.php/br/items/releasing-boost-dependency-ana… kind regards, Jens Weller

1 0

Compiling Boost 1.54.0 for the iOS simulator (6.1) on OS X 10.8.4
by Dan Leach 09 Aug '13

09 Aug '13

Hello, I am new to the Boost mailing list so please let me know if this is not the appropriate place to discuss my problem. I have successfully managed to compile several of the Boost libraries (regex, filesystem, thread) and create a framework for use with OS X, iOS, and the iOS simulator using this script: https://github.com/wuhao5/boost/blob/188e905626dbd522f65950102ed0c8ce77cb28… on Peter Goodliffe's work here: http://goodliffe.blogspot.com/2010/09/building-boost-framework-for-ios-ipho… Boost 1.53.0 using clang and linking against libc++. The script fails when compiling for the iOS simulator with Boost 1.54.0 with the following error:/bin/sh: line 1: -ftemplate-depth-128: command not found. I did quite a bit of digging and here is what I know: * In boost.sh, the iOS simulator bjam and compiler options include "-arch i386" * The release notes for Boost 1.54.0 state that "Boost no longer supports the 80386 target CPU" * The Boost 1.53.0 libraries mentioned above appear to compile fine for OS X, iOS, and the iOS simulator using the same script * The Boost 1.54.0 libraries appear to compile fine for OS X and iOS itself using the same script Here is what I think I know: * When passing "-d13" to bjam (to generate debug information), it appears that the variable "CONFIG_COMMAND" is not being set when building for the iOS simulator. For the other configurations, this variable is set to the path to clang and includes the compiler options. For the simulator, it is empty which I believe leads to the error above as when bjam attempts to run the compiler using this variable, it instead runs some appended options which begin with the template depth. Therefore, the CONFIG_COMMAND variable does not appear to be being set properly causing the failure. I do not know why it's not being set as I don't know enough about the Boost build system to sift through the rest of the output generated with the "-d13" flag. I suspect an incompatibility with the "-arch i386" option was introduced with 1.54.0. I've tried changing this to "-arch i486" and "-arch x86_32" but the same problem occurs. Has anyone else had success with this? I'd like to use Boost.Log which is why I'm even attempting this with 1.54. Thanks!

1 1

[program_options] Parsing config files
by Michael Powell 08 Aug '13

08 Aug '13

Hello, Need a little help grasping the nature of parse_config_file. The example: http://www.boost.org/doc/libs/1_54_0/doc/html/program_options/tutorial.html… also seems a bit lacking: says that there are two calls to store, but I only see one. Maybe the example has not been updated? What I need to know is, if we specify some options, do we specify the same option names in a config file? So that if we have an option on the command like like: --my-option-name 1234, then the corresponding line in the config file would go: my-option-name=1234 ? Thank you. Regards, Michael Powell

2 9

Re: [Boost-users] Library Status
by Malysa, Eric (GE Healthcare, consultant) 07 Aug '13

07 Aug '13

Hello, I am trying to run some regression tests, and then create a status report following the documentation provided here: http://www.boost.org/doc/libs/1_54_0/tools/regression/doc/library_status.ht… I am using version Boost Version 1.53.0 on Suse 11 patch 1. PETMRds32-2:~> cat /etc/*-release LSB_VERSION="core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-ia32:core-3.2-ia32:core-4.0-ia32" SUSE Linux Enterprise Server 11 (i586) VERSION = 11 PATCHLEVEL = 1 PETMRds32-2:~> 1. Start from your command line environment. 2. set the current directory to:../libs/<library name>/test 3. Invoke one of the following: o ../../../tools/regression/src/library_test (*nix). o ..\..\..\tools\regression\src\library_test (windows). 4. This will display short help message describing the how to set the command line arguments for the compilers and variants you want to appear in the final table. 5. Setting these arguments requires rudimentary knowledge of bjam usage. Hopefully, if you've arrived at this page you've gained the required knowledge during the installation and library build process. 6. Rerun the above command with the argument set accordingly. 7. When the command terminates, there should be a file named "library_status.html" in the current directory. 8. Display this file with any web browser. The problem is when it calls library_status there is an error about a invalid bin directory. I would like to know what it is looking for? -----Original Message----- From: Boost-users [mailto:boost-users-bounces@lists.boost.org] On Behalf Of boost-users-request(a)lists.boost.org Sent: Wednesday, August 07, 2013 6:50 AM To: boost-users(a)lists.boost.org Subject: Boost-users Digest, Vol 3529, Issue 1 Send Boost-users mailing list submissions to boost-users(a)lists.boost.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.boost.org/mailman/listinfo.cgi/boost-users or, via email, send a message with subject or body 'help' to boost-users-request(a)lists.boost.org You can reach the person managing the list at boost-users-owner(a)lists.boost.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Boost-users digest..." Today's Topics: 1. Library Status (Malysa, Eric (GE Healthcare, consultant)) 2. Re: [Coroutine] Unresolved externals with VC2012 (Szymon Gatner) 3. [proposal] are_same on type_traits (skypjack) 4. Re: Boost.Variant broke in 1.54? (Edward Diener) 5. Re: Library Status (Igor R) 6. [msm] exit points & favor compile time policy (Alexander Mingalev) 7. Re: cannot use boost asio 1.54 with visual studio 2008 (Antony Polukhin) ---------------------------------------------------------------------- Message: 1 Date: Tue, 6 Aug 2013 20:32:43 +0000 From: "Malysa, Eric (GE Healthcare, consultant)" <eric.malysa(a)ge.com> To: "boost-users(a)lists.boost.org" <boost-users(a)lists.boost.org> Cc: "Dukle, Kapil \(GE Healthcare\)" <Kapil.Dukle(a)med.ge.com> Subject: [Boost-users] Library Status Message-ID: <55788829594D1745B52483994D25C5250155C6(a)ALPURAPA08.e2k.ad.ge.com> Content-Type: text/plain; charset="us-ascii" Hello, What conditions for a "bin" directory is library_status.cpp looking for? The documentation seems either inadequate or wrong. I am getting an error: **** exception(205): std::string: invalid bin directory structure ******** errors detected; see standard output for details ******** I am using compiler gcc 4.3 Your help is very much appreciated. Thanks, Eric Malysa

2 1

Serialising templates with smart pointers....
by Marcus MacWilliam 07 Aug '13

07 Aug '13

I have the following: _*1. An interface:*_ class IDeviceClientRequest { public: /** Empty virtual destructor. */ virtual ~IDeviceClientRequest() {} /** Virtual methods to invoke on the smart pointer. */ virtual EClientMessageIdentifier getMessageId() = 0; }; // IDeviceClientRequest _*2. A template class:*_ template<class Request> class CDeviceClientRequest : public IDeviceClientRequest { public: // ... I have excluded the majority of the code for conciseness // ... private: /** * Serialize or de-serialize the object. * * @param archive * When the class Archive corresponds to an output archive, the * & operator is defined similar to <<. Likewise, when the class Archive * is a type of input archive the & operator is defined similar to >>. * @param version * Version identifier for backwards compatibility between objects. */ friend class boost::serialization::access; template<class Archive> void serialize(Archive &archive, const unsigned int version) { archive & m_iMessageId & m_oData; } }; // CDeviceClientRequest _*3. A method to serialize from a file:*_ // Server receieves a request. void devices::comms::CDeviceCommunication::receiveRequest( std::unique_ptr<devices::request::IDeviceClientRequest>& request) { // Open the file for read. std::ifstream file(getFilename().c_str()); // Use boost to serialize the object from the file. boost::archive::binary_iarchive archive(file); archive >> *request; // Close the file. file.close(); } What do I need to add to the interface class to stop the following compile error: error C2039: 'serialize' : is not a member of 'devices::request::IDeviceClientRequest' Thanks, -- Marcus A.T MacWilliam, Senior Software Engineer. 1Spatial Ltd. Tel: +44 (0)1223 420414, ext 2289, Mob: +44 (0)7803 706597. E-Mail: marcus.macwilliam(a)1spatial.com Skype:marcus.macwilliam Tennyson House, 7 Cambridge Business Park, Cambridge, CB4 0WZ. Registered in England No. 4785688, VAT Reg. No. 135539408. Website: http://www.1spatial.com/ E-mail: info(a)1spatial.com

1 0