WebServer upload is stuck with latest git but ok in 2.7.4 and previous

[luc-github]

• [x ] This issue complies with the issue POLICY doc.
• [x ] I have read the documentation at readthedocs and the issue is not addressed there.
• [ x] I have tested that the issue is present in current master branch (aka latest git).
• [x ] I have searched the issue tracker for a similar issue.
• [ N/A] If there is a stack dump, I have decoded it.
• [x ] I have filled out all fields below.

Platform

• Hardware: [ESP-12|]
• Core Version: latest git 31/12/2020
• Development Env: Arduino IDE|
• Operating System: Windows

Settings in IDE

• Module: Generic ESP8266 Module
• Flash Mode: dio
• Flash Size: 4MB/2MB FS
• lwip Variant: v2 Lower Memory
• Reset Method: nodemcu
• Flash Frequency: [40Mhz]
• CPU Frequency: 160MHz
• Upload Using: SERIAL
• Upload Speed: 115200

Problem Description

when using latest git esp core, uploading file to webserver the upload is stuck because curl is waiting an ack that the webserver do not send

C:\Users\luc\Documents\GitHub\ESP3D-WEBUI>curl -v -F "file=@nodemon.json" http://192.168.1.75/upload
*   Trying 192.168.1.75...
* TCP_NODELAY set
* Connected to 192.168.1.75 (192.168.1.75) port 80 (#0)
> POST /upload HTTP/1.1
> Host: 192.168.1.75
> User-Agent: curl/7.55.1
> Accept: */*
> Content-Length: 265
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------6509fe3b11f716de
>
* Done waiting for 100-continue

same code using espcore 2.7.4 and older also are working fine and were working as I used it

C:\Users\luc\Documents\GitHub\ESP3D-WEBUI>curl -v -F "file=@nodemon.json" http://192.168.1.75/upload
*   Trying 192.168.1.75...
* TCP_NODELAY set
* Connected to 192.168.1.75 (192.168.1.75) port 80 (#0)
> POST /upload HTTP/1.1
> Host: 192.168.1.75
> User-Agent: curl/7.55.1
> Accept: */*
> Content-Length: 265
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------24d01294b659dad6
>
* Done waiting for 100-continue
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 9
< Connection: close
<
ok boomer* Closing connection 0

MCVE Sketch

I use standard API

webserver.on ("/upload", HTTP_ANY, handleMKSUpload, MKSFileupload);

So FSBrowser.ino should do same

That was the last issue from me for 2020 🎆 🎉
Happy new year 2021 🥳 - a big thank you for this year to the dev team and continue your great job ^_^ 🎈

[luc-github]

Issue start with this commit #7728

[earlephilhower]

Can you please make a MCVE to repro your failure? The FSBrowser upload command is working AFAICT, with me and the user who found the issue both seeing good results so there's something different going on here.

Since HTTP multipart is kind of bogus and depends on in-stream data signaling, it may be related to the actual file contents, so please include a sample file we can curl upload, too. That way it's nice and reproducible.

[earlephilhower]

earle@server:~$ curl -v -F "file=@xfer.pdf" http://192.168.1.133/edit
*   Trying 192.168.1.133...
* TCP_NODELAY set
* Connected to 192.168.1.133 (192.168.1.133) port 80 (#0)
> POST /edit HTTP/1.1
> Host: 192.168.1.133
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 41912
> Content-Type: multipart/form-data; boundary=------------------------e751d76c9609d040
> Expect: 100-continue
> 
* Done waiting for 100-continue
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 0
< Connection: keep-alive
< Keep-Alive: timeout=2000
< 
* Connection #0 to host 192.168.1.133 left intact

[luc-github]

webserver.zip
Here the MCVE
I use only text files - they are enclosed

may be the curl version under windows is different ?

C:\Users\luc\Desktop\webserver>curl --version
curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL
Release-Date: 2017-11-14, security patched: 2019-11-05
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL

[luc-github]

I have updated curl and behavior is same but curl output is just different, anyway still stuck

C:\Users\luc\Desktop\webserver>curl --version
curl 7.74.0 (x86_64-pc-win32) libcurl/7.74.0 OpenSSL/1.1.1i (Schannel) zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 WinIDN libssh2/1.9.0 nghttp2/1.42.0
Release-Date: 2020-12-09
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile MultiSSL NTLM SPNEGO SSL SSPI TLS-SRP Unicode UnixSockets alt-svc brotli libz zstd

C:\Users\luc\Desktop\webserver>curl -v -L -F "file=@gpl.txt" http://192.168.1.75/upload
*   Trying 192.168.1.75:80...
* Connected to 192.168.1.75 (192.168.1.75) port 80 (#0)
> POST /upload HTTP/1.1
> Host: 192.168.1.75
> User-Agent: curl/7.74.0
> Accept: */*
> Content-Length: 36006
> Content-Type: multipart/form-data; boundary=------------------------25b5e4a2091a7b92
>
* We are completely uploaded and fine

the upload still does not end

Starting upload
Filename gpl.txt
.................

[earlephilhower]

Thanks, I can repro it with your sketch and file. Curl on Windows isn't any different, but your EOL chars are, and I think that's where the issue lies. Having \r\n as the last 2 chars seems to throw off the FSM.

[luc-github]

one interresting thing is: like you, under linux it is working

luc@luc-VirtualBox:~$ curl -1 -v -L -F "file=@gpl.txt" http://192.168.1.75/upload
*   Trying 192.168.1.75...
* TCP_NODELAY set
* Connected to 192.168.1.75 (192.168.1.75) port 80 (#0)
> POST /upload HTTP/1.1
> Host: 192.168.1.75
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 35332
> Content-Type: multipart/form-data; boundary=------------------------df4dde41d48d0a33
> Expect: 100-continue
> 
* Done waiting for 100-continue
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 9
< Connection: keep-alive
< Keep-Alive: timeout=2000
< 
* Connection #0 to host 192.168.1.75 left intact
ok boomerluc@luc-VirtualBox:~$ 

EDIT : may be because not same gpl.txt