Default to WHATWG URL parser in http.request (and friends)

[TimothyGu]

• Version: master
• Platform: all
• Subsystem: http

Currently, http.request('http://brave.com%60x.code-fu.org/') requests http://brave.com/%60x.code-fu.org/ rather than http://brave.com`x.code-fu.org/. This behavior deviates from the behavior standardized in WHATWG URL Standard and used in browsers, and have caused dangerous security implications for downstream embedders (see talk by @diracdeltas).

This is due to the http.request function using the legacy url.parse function rather than the new WHATWG-compliant URL parser. We should switch the URL parser used for string-typed argument to the standard-complaint parser.

This switch will surely have compatibility implications, but I doubt it will cause major breakage since most request-style libraries pass in an object instead of a string as the first argument anyway.

/cc @annevk @BrendanEich (See original tweet.)

[jasnell]

+1 on this. This was my intent originally when introducing the WHATWG URL parser but I wanted to give the new implementation time to be stable before introducing the change.

There are some hidden gotchas in here. The net code is currently written to assume url.parse() is used and is currently set up at multiple levels to consume the parsed object it provides as opposed to the actual URL class produced by the new parser.

[TimothyGu]

@jasnell I was thinking we could just convert the created URL object to the internally used Url format right after parsing, using the conversion function already available to us. That wouldn’t cause as much churn.

[Hackzzila]

I have implemented this in #20270. I think it would be good improvement to v11. By that time the WHATWG URL parser will have had about 18 months to stabilize.