Boost.URL Proposal max_size
Hi! We're deciding on the max size for URLs in our Boost.URL proposal. The spec doesn't suggest a limit but the reasonable trade-off <https://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers> is: - 2^16 chars: More than all browsers, servers, and search engines support (~2000 chars). More than the HTTP minimum of 8000 octets. - 2^32 chars: "414 Too long request" in practice everywhere. Maybe supports experiments using URLs as a container for something else. Any suggestions? -- Alan Freitas https://github.com/alandefreitas
On Thu, Mar 24, 2022 at 5:13 PM Alan de Freitas via Boost <boost@lists.boost.org> wrote:
We're deciding on the max size for URLs in our Boost.URL proposal. The spec doesn't suggest a limit but the reasonable trade-off <https://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers> is:
- 2^16 chars: More than all browsers, servers, and search engines support (~2000 chars). More than the HTTP minimum of 8000 octets. - 2^32 chars: "414 Too long request" in practice everywhere. Maybe supports experiments using URLs as a container for something else.
Any suggestions?
This recent Fossil-SCM thread (about althttpd) might be relevant: https://fossil-scm.org/forum/forumpost/7f445ff3679a8fd6 I think personally 8KB or 16KB is enough. Your suggested 64KB is OK too, I guess. But your 4GB proposal seems way too big. I wouldn't go above 2^20 = 1MB myself, FWIW. --DD
On Thu, Mar 24, 2022 at 9:24 AM Dominique Devienne via Boost <boost@lists.boost.org> wrote:
I think personally 8KB or 16KB is enough. Your suggested 64KB is OK too, I guess. But your 4GB proposal seems way too big. I wouldn't go above 2^20 = 1MB myself, FWIW. --DD
if we go over 64kb then the next possible size is 2^32-1 bytes, as the question is related to sizing some offsets that are used in lookup tables. Of course, if the limit is 2GB or 4GB the user can always impose a lower limit. Thanks
On 3/24/22 19:11, Alan de Freitas via Boost wrote:
Hi!
We're deciding on the max size for URLs in our Boost.URL proposal. The spec doesn't suggest a limit but the reasonable trade-off <https://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers> is:
- 2^16 chars: More than all browsers, servers, and search engines support (~2000 chars). More than the HTTP minimum of 8000 octets. - 2^32 chars: "414 Too long request" in practice everywhere. Maybe supports experiments using URLs as a container for something else.
Any suggestions?
I'd prefer 2^16, especially if it allows to reduce the URI object size (meaning the object itself, irrespective of the URI string body). Excessive limits on the URI size may be a source of resource exhaustion vulnerabilities when the URI is obtained from an external source.
On 24.03.22 17:11, Alan de Freitas via Boost wrote:
Hi!
We're deciding on the max size for URLs in our Boost.URL proposal. The spec doesn't suggest a limit but the reasonable trade-off <https://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers> is:
- 2^16 chars: More than all browsers, servers, and search engines support
That's not true. Both Firefox and Chromium support multi-megabyte embedded images using the data URI scheme. -- Rainer Deyke (rainerd@eldwood.com)
Hi Rainer, You're correct. Data URIs probably justify 2^32 chars. In some contexts, there's a max size of 4GB for these: https://caniuse.com/datauri []s, On Fri, 25 Mar 2022 at 04:22, Rainer Deyke via Boost <boost@lists.boost.org> wrote:
On 24.03.22 17:11, Alan de Freitas via Boost wrote:
Hi!
We're deciding on the max size for URLs in our Boost.URL proposal. The spec doesn't suggest a limit but the reasonable trade-off < https://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-u...
is:
- 2^16 chars: More than all browsers, servers, and search engines support
That's not true. Both Firefox and Chromium support multi-megabyte embedded images using the data URI scheme.
-- Rainer Deyke (rainerd@eldwood.com)
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-- Alan Freitas https://github.com/alandefreitas
Alan de Freitas wrote:
We're deciding on the max size for URLs in our Boost.URL proposal.
I have a webmail system that converts email attachments (e.g. images) to data: URLs; for that, 2^16 would be too small. I've recently had an application where a request was logically a GET, but had to be implemented as a POST (with the data in the request body) because the parameter data size could exceed the URL length limit imposed by some part of the system; in this case the max length would have been more than 2^16 but less than 2^20 bytes I think. On that basis I'd suggest that you need to store 32-bit offsets. Note that I'm not convinced that your approach of parsing and storing all of these offsets when the URL object is created is necessarily the best approach. An alternative would be to parse to determine that the string is a valid URL but store few or none of the offsets; then re-scan to find the URL parts if needed later. There's an obvious trade off between object size and accessor complexity. Regards, Phil.
participants (6)
-
Alan de Freitas -
Andrey Semashev -
Dominique Devienne -
Phil Endecott -
Rainer Deyke -
Vinnie Falco