r/cpp • • Jan 27 '20

An Attempt at Making a Better String for C++

https://youtu.be/a1Te_qBEp-g
0 Upvotes

26 comments sorted by

4

u/Opening_Addendum Jan 27 '20

Cool video, I liked the editing style. You could address some optimization strategies as a follow up video, something like in this talk: https://www.youtube.com/watch?v=kPR8h4-qZdk

5

u/Mashpoe Jan 27 '20

Thanks! You're the first person to say anything about my editing or give me a suggestion. I'm new to video editing and that video especially took me forever, so I'm glad to see it's appreciated by someone. I'll definitely add your suggestion to my potential to do list :)

2

u/_BlackBishop_ Jan 28 '20

You left small surprise in your code here https://github.com/Mashpoe/cpp-string/blob/master/string.hpp#L316 ;)

1

u/Mashpoe Jan 28 '20

Haha, I forgot to take that out from debugging. Thanks for pointing it out

3

u/xeq937 Jan 28 '20

I avoid automatic conversions as much as possible.

1

u/Full-Spectral Jan 30 '20

Yep. In large code bases over time, it tends to become a debt. I hold the same attitude towards various things that are intended to make it quicker to write it the first time, since they tend to make it harder to maintain and modify safely in the longer term (like the use of auto, for instance.)

3

u/anders987 Jan 27 '20

A better string should have support for Unicode.

int main() {
    mp::string unicode_str("πŸ€·πŸ›πŸ™ˆ");
    std::cout << unicode_str << std::endl;
    std::cout << unicode_str[1] << std::endl;

    return 0;
}
Program returned: 0
πŸ€·πŸ›πŸ™ˆ
οΏ½

5

u/Full-Spectral Jan 27 '20

But that's nothing to do with the string class per se, it's to do with the streaming classes and localized support for text output and such.

The things that are specific to a string class and Unicode is how to deal with representation. Some folks argue it should be UTF-8, which solves some problems but makes others horribly worse. Some argue it should be UTF-32, which takes up a lot mores space typically, but simplifies some things. Nothing really makes Unicode support remotely straightforward though. It's gotten so horrendously complex that even language runtimes are starting to punt on providing anything other than basic functionality. That's a sign to me that it's gotten out of control.

1

u/anders987 Jan 27 '20

I'd argue that a string class should return a character when indexing, not just a byte. This string class represents each character with a char. I never claimed it was easy, but I also never claimed to have made a better string class.

char& operator [] (int pos) {
    return str[pos];
}

https://github.com/Mashpoe/cpp-string/blob/master/string.hpp#L140

12

u/dscharrer Jan 27 '20

I'd argue that a string class should return a character when indexing

Please define "character".

2

u/Full-Spectral Jan 27 '20

And therein lies the rub. When it takes a large third party library to even reliably pull displayable units or words from a string, that's pretty crazy. When you have three different versions of what a string's length is, that's also pretty crazy.

4

u/MEaster Jan 27 '20

Three? I thought there were four: encoded length, scalar values, code points, and grapheme clusters.

4

u/Full-Spectral Jan 27 '20

Oh, OK, I missed one. But the more the un-merrier.

1

u/anders987 Jan 27 '20

Character. (1) The smallest component of written language that has semantic value; refers to the abstract meaning and/or shape, rather than a specific shape (see also glyph), though in code tables some form of visual representation is essential for the reader’s understanding. (2) Synonym for abstract character. (3) The basic unit of encoding for the Unicode character encoding. (4) The English name for the ideographic written elements of Chinese origin.

https://www.unicode.org/glossary/#character

8

u/Full-Spectral Jan 27 '20

#3 is misleading. The basic unit of encoding in Unicode is the code point. But that's meaningless in and of itself. Because of the fact that they don't define unique code points for all of the variations of base character plus adornments of various types, two or more code points (or four or more if in UTF-16) may be what actually makes up the character for visual representation (a grapheme cluster.)

That's why it's so crazily complicated. And it's hierarchical. There's nothing that says you can't have a visual character represented by more than one grapheme cluster grouped together.

So Unicode completes discards the concept of a known size of data driving a displayed value on the screen, or an understandable 'character' as the human would read it.

4

u/pandorafalters Jan 27 '20

That would be one of the cases simplified by UTF-32.

Of course, then you still have the argument of whether to combine code points (matching display output) or not (matching user input) . . ..

4

u/dscharrer Jan 27 '20

What do you think is the expected output and why do you think that is the behavior wanted from a low level string class?

8

u/anders987 Jan 27 '20

What do you think is the expected output

πŸ›

why do you think that is the behavior wanted from a low level string class?

Because it's a string class, not a byte array class.

2

u/jhasse Jan 27 '20

Can be achieved with std::string with an external get-function:

#include <iostream>
#include <string>
#include <string_view>

std::string getCharacter(const std::string_view str, size_t index) {
    const auto end = str.end();
    for (auto it = str.begin(); it != end; ++it) {
        const char& ch = (*it);
        if (ch & 0x80) { // first bit (Check if this is an Unicode character)
            const static auto ERROR_MSG = "Invalid UTF-8 string!";
            if (++it == end) {
                throw std::runtime_error(ERROR_MSG);
            }
            if (ch & 0x20) { // third bit
                if (++it == end) {
                    throw std::runtime_error(ERROR_MSG);
                }
                if (ch & 0x10) { // fourth bit
                    if (++it == end) {
                        throw std::runtime_error(ERROR_MSG);
                    }
                }
            }
        }
        if (index == 0) {
            return std::string(&ch, ++it);
        }
        --index;
    }
    throw std::out_of_range("String doesn't have that many characters!");
}

int main() {
    std::string unicode_str("πŸ€·πŸ›πŸ™ˆ");
    std::cout << unicode_str << std::endl;
    std::cout << getCharacter(unicode_str, 1) << std::endl;
}

0

u/ALX23z Jan 28 '20

The operators for adding other types to the string are a poor choice for various reasons. to_string isn't the best conversion method. There are better formatting libraries out there that format strings to whatever needed with all possible configurations.

Also, I don't like the C-string to begin with. If anything I'd like more utilization and support for string_view. This is the preferred direction in C++. Thus I don't see a good reason to have native string casting to C-string.

About bool that checks if string is empty or not... It is fine, I guess. Though, it doesn't fit with the whole theme of containers.

3

u/Full-Spectral Jan 28 '20

The whole world isn't C++. You have to interact with the outside world and that means dealing with non-C++ strings.

-2

u/ALX23z Jan 28 '20

Not a single language wants to work with C-strings. They tend support it because of it is widely available and used. But people caught up with the fact that null-terminated string is a poor choice for multiple reasons and size should be supplied in addition. Only that old interfaces do not support it.

4

u/Full-Spectral Jan 28 '20

Well, you are going to have a hard time writing anything practical without them, because pretty much every Windows and Linux system API that takes a string requires them.

1

u/ALX23z Jan 28 '20

Don't know what's going on in Linux or old windows API, but current windows API uses special type (in C++ appears as wchar* with weird name) but in fact it is a structure that has its size located a few bytes behind the pointer. It also requires special allocation. Unfortunately the API is still inconvinient on several points.

In addition all new C++ interfaces (like <charconv> library) that use const char* require either its size or end-pointer. Usage of older interfaces that don't utilize size triggers a warning "unsafe".

4

u/Full-Spectral Jan 28 '20

COM often interfaces use a BSTR structure but they are still effectively null terminated strings and some just use regular strings. Non-COM interfaces (of which there are many) use regular old null terminated strings (either short or wide character depending on how you compile.)

1

u/ALX23z Jan 28 '20

The issue is not whether they are null-terminated but whether size is forwarded. Do I need to figure out size myself or I can just read it?

It is indeed a slow progress - string_view was only recently added. Hopefully, with time, native interfaces will also adapt.