204 lines
12 KiB
HTML
204 lines
12 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
|
||
<html>
|
||
<head>
|
||
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
|
||
<title>Design Discussion</title>
|
||
<link rel="stylesheet" href="../../../doc/src/boostbook.css" type="text/css">
|
||
<meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
|
||
<link rel="home" href="../index.html" title="The Boost C++ Libraries BoostBook Documentation Subset">
|
||
<link rel="up" href="../program_options.html" title="Chapter 31. Boost.Program_options">
|
||
<link rel="prev" href="howto.html" title="How To">
|
||
<link rel="next" href="s06.html" title="Acknowledgements">
|
||
</head>
|
||
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
|
||
<table cellpadding="2" width="100%"><tr>
|
||
<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../boost.png"></td>
|
||
<td align="center"><a href="../../../index.html">Home</a></td>
|
||
<td align="center"><a href="../../../libs/libraries.htm">Libraries</a></td>
|
||
<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
|
||
<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
|
||
<td align="center"><a href="../../../more/index.htm">More</a></td>
|
||
</tr></table>
|
||
<hr>
|
||
<div class="spirit-nav">
|
||
<a accesskey="p" href="howto.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../program_options.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="s06.html"><img src="../../../doc/src/images/next.png" alt="Next"></a>
|
||
</div>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
|
||
<a name="program_options.design"></a>Design Discussion</h2></div></div></div>
|
||
<div class="toc"><dl class="toc"><dt><span class="section"><a href="design.html#program_options.design.unicode">Unicode Support</a></span></dt></dl></div>
|
||
<p>This section focuses on some of the design questions.
|
||
</p>
|
||
<div class="section">
|
||
<div class="titlepage"><div><div><h3 class="title">
|
||
<a name="program_options.design.unicode"></a>Unicode Support</h3></div></div></div>
|
||
<p>Unicode support was one of the features specifically requested
|
||
during the formal review. Throughout this document "Unicode support" is
|
||
a synonym for "wchar_t" support, assuming that "wchar_t" always uses
|
||
Unicode encoding. Also, when talking about "ascii" (in lowercase) we'll
|
||
not mean strict 7-bit ASCII encoding, but rather "char" strings in local
|
||
8-bit encoding.
|
||
</p>
|
||
<p>
|
||
Generally, "Unicode support" can mean
|
||
many things, but for the program_options library it means that:
|
||
|
||
</p>
|
||
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
|
||
<li class="listitem"><p>Each parser should accept either <code class="computeroutput">char*</code>
|
||
or <code class="computeroutput">wchar_t*</code>, correctly split the input into option
|
||
names and option values and return the data.
|
||
</p></li>
|
||
<li class="listitem"><p>For each option, it should be possible to specify whether the conversion
|
||
from string to value uses ascii or Unicode.
|
||
</p></li>
|
||
<li class="listitem">
|
||
<p>The library guarantees that:
|
||
</p>
|
||
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; ">
|
||
<li class="listitem"><p>ascii input is passed to an ascii value without change
|
||
</p></li>
|
||
<li class="listitem"><p>Unicode input is passed to a Unicode value without change</p></li>
|
||
<li class="listitem"><p>ascii input passed to a Unicode value, and Unicode input
|
||
passed to an ascii value will be converted using a codecvt
|
||
facet (which may be specified by the user).
|
||
</p></li>
|
||
</ul></div>
|
||
<p>
|
||
</p>
|
||
</li>
|
||
</ul></div>
|
||
<p>
|
||
</p>
|
||
<p>The important point is that it's possible to have some "ascii
|
||
options" together with "Unicode options". There are two reasons for
|
||
this. First, for a given type you might not have the code to extract the
|
||
value from Unicode string and it's not good to require that such code be written.
|
||
Second, imagine a reusable library which has some options and exposes
|
||
options description in its interface. If <span class="emphasis"><em>all</em></span>
|
||
options are either ascii or Unicode, and the library does not use any
|
||
Unicode strings, then the author is likely to use ascii options, making
|
||
the library unusable inside Unicode
|
||
applications. Essentially, it would be necessary to provide two versions
|
||
of the library -- ascii and Unicode.
|
||
</p>
|
||
<p>Another important point is that ascii strings are passed though
|
||
without modification. In other words, it's not possible to just convert
|
||
ascii to Unicode and process the Unicode further. The problem is that the
|
||
default conversion mechanism -- the <code class="computeroutput">codecvt</code> facet -- might
|
||
not work with 8-bit input without additional setup.
|
||
</p>
|
||
<p>The Unicode support outlined above is not complete. For example, we
|
||
don't support Unicode option names. Unicode support is hard and
|
||
requires a Boost-wide solution. Even comparing two arbitrary Unicode
|
||
strings is non-trivial. Finally, using Unicode in option names is
|
||
related to internationalization, which has it's own
|
||
complexities. E.g. if option names depend on current locale, then all
|
||
program parts and other parts which use the name must be
|
||
internationalized too.
|
||
</p>
|
||
<p>The primary question in implementing the Unicode support is whether
|
||
to use templates and <code class="computeroutput">std::basic_string</code> or to use some
|
||
internal encoding and convert between internal and external encodings on
|
||
the interface boundaries.
|
||
</p>
|
||
<p>The choice, mostly, is between code size and execution
|
||
speed. A templated solution would either link library code into every
|
||
application that uses the library (thereby making shared library
|
||
impossible), or provide explicit instantiations in the shared library
|
||
(increasing its size). The solution based on internal encoding would
|
||
necessarily make conversions in a number of places and will be somewhat slower.
|
||
Since speed is generally not an issue for this library, the second
|
||
solution looks more attractive, but we'll take a closer look at
|
||
individual components.
|
||
</p>
|
||
<p>For the parsers component, we have three choices:
|
||
</p>
|
||
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
|
||
<li class="listitem"><p>Use a fully templated implementation: given a string of a
|
||
certain type, a parser will return a <code class="computeroutput"><a class="link" href="reference.html#boost.program_options.parsed_options">parsed_options</a></code> instance
|
||
with strings of the same type (i.e. the <code class="computeroutput"><a class="link" href="reference.html#boost.program_options.parsed_options">parsed_options</a></code> class
|
||
will be templated).</p></li>
|
||
<li class="listitem"><p>Use internal encoding: same as above, but strings will be converted to and
|
||
from the internal encoding.</p></li>
|
||
<li class="listitem"><p>Use and partly expose the internal encoding: same as above,
|
||
but the strings in the <code class="computeroutput"><a class="link" href="reference.html#boost.program_options.parsed_options">parsed_options</a></code> instance will be in the
|
||
internal encoding. This might avoid a conversion if
|
||
<code class="computeroutput"><a class="link" href="reference.html#boost.program_options.parsed_options">parsed_options</a></code> instance is passed directly to other components,
|
||
but can be also dangerous or confusing for a user.
|
||
</p></li>
|
||
</ul></div>
|
||
<p>
|
||
</p>
|
||
<p>The second solution appears to be the best -- it does not increase
|
||
the code size much and is cleaner than the third. To avoid extra
|
||
conversions, the Unicode version of <code class="computeroutput"><a class="link" href="reference.html#boost.program_options.parsed_options">parsed_options</a></code> can also store
|
||
strings in internal encoding.
|
||
</p>
|
||
<p>For the options descriptions component, we don't have much
|
||
choice. Since it's not desirable to have either all options use ascii or all
|
||
of them use Unicode, but rather have some ascii and some Unicode options, the
|
||
interface of the <code class="computeroutput"><a class="link" href="../boost/program_options/value_semantic.html" title="Class value_semantic">value_semantic</a></code> must work with both. The only way is
|
||
to pass an additional flag telling if strings use ascii or internal encoding.
|
||
The instance of <code class="computeroutput"><a class="link" href="../boost/program_options/value_semantic.html" title="Class value_semantic">value_semantic</a></code> can then convert into some
|
||
other encoding if needed.
|
||
</p>
|
||
<p>For the storage component, the only affected function is <code class="computeroutput"><a class="link" href="../boost/program_options/store_1_3_32_9_11_1_1_5.html" title="Function store">store</a></code>.
|
||
For Unicode input, the <code class="computeroutput"><a class="link" href="../boost/program_options/store_1_3_32_9_11_1_1_5.html" title="Function store">store</a></code> function should convert the value to the
|
||
internal encoding. It should also inform the <code class="computeroutput"><a class="link" href="../boost/program_options/value_semantic.html" title="Class value_semantic">value_semantic</a></code> class
|
||
about the used encoding.
|
||
</p>
|
||
<p>Finally, what internal encoding should we use? The
|
||
alternatives are:
|
||
<code class="computeroutput">std::wstring</code> (using UCS-4 encoding) and
|
||
<code class="computeroutput">std::string</code> (using UTF-8 encoding). The difference between
|
||
alternatives is:
|
||
</p>
|
||
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
|
||
<li class="listitem"><p>Speed: UTF-8 is a bit slower</p></li>
|
||
<li class="listitem"><p>Space: UTF-8 takes less space when input is ascii</p></li>
|
||
<li class="listitem"><p>Code size: UTF-8 requires additional conversion code. However,
|
||
it allows one to use existing parsers without converting them to
|
||
<code class="computeroutput">std::wstring</code> and such conversion is likely to create a
|
||
number of new instantiations.
|
||
</p></li>
|
||
</ul></div>
|
||
<p>
|
||
There's no clear leader, but the last point seems important, so UTF-8
|
||
will be used.
|
||
</p>
|
||
<p>Choosing the UTF-8 encoding allows the use of existing parsers,
|
||
because 7-bit ascii characters retain their values in UTF-8,
|
||
so searching for 7-bit strings is simple. However, there are
|
||
two subtle issues:
|
||
</p>
|
||
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
|
||
<li class="listitem"><p>We need to assume the character literals use ascii encoding
|
||
and that inputs use Unicode encoding.</p></li>
|
||
<li class="listitem"><p>A Unicode character (say '=') can be followed by 'composing
|
||
character' and the combination is not the same as just '=', so a
|
||
simple search for '=' might find the wrong character.
|
||
</p></li>
|
||
</ul></div>
|
||
<p>
|
||
Neither of these issues appear to be critical in practice, since ascii is
|
||
almost universal encoding and since composing characters following '=' (and
|
||
other characters with special meaning to the library) are not likely to appear.
|
||
</p>
|
||
</div>
|
||
</div>
|
||
<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
|
||
<td align="left"></td>
|
||
<td align="right"><div class="copyright-footer">Copyright © 2002-2004 Vladimir Prus<p>Distributed under the Boost Software License, Version 1.0.
|
||
(See accompanying file <code class="filename">LICENSE_1_0.txt</code> or copy at
|
||
<a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
|
||
</p>
|
||
</div></td>
|
||
</tr></table>
|
||
<hr>
|
||
<div class="spirit-nav">
|
||
<a accesskey="p" href="howto.html"><img src="../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../program_options.html"><img src="../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="s06.html"><img src="../../../doc/src/images/next.png" alt="Next"></a>
|
||
</div>
|
||
</body>
|
||
</html>
|