Re: [U-Boot] [PATCH v10 3/3] Adding wget
From: Simon Glass sjg@chromium.org To: Duncan Hare dh@synoia.com Sent: Sunday, April 22, 2018 1:10 PM Subject: Re: [PATCH v10 3/3] Adding wget
Hi Duncan,
On 17 April 2018 at 15:58, Duncan Hare dh@synoia.com wrote:
From: Simon Glass sjg@chromium.org It has been through patman, and the only errors flagged a packed structures, necessary for packed headers.
It should be possible in the Python test to enable an http server on localhost with a few lines of code, and then connect to it in the test?
Yes server at port 80. The server can be tested with the wget command which can be installed on linux.I doubt that loop-back like this will produce the scrambling of packet order which is a feature of push down stacks for packet queuesin the internet. The pi is easy to overrun when testing on a low latency network, because the TCP send window size is defined by the number ofnet_rx_buffers, which is controlled the CONFIG_SYS_RX_ETH_BUFFER in include.net.h, which sets PKTBUFSRX at the beginning of include/net.h, which in-turn defines a buffer pool in net/net.c, array named net_pkt_buf. Hence my comment in a different thread about buffering on the pi. Few of the socs appear to use net_pkt_buf buffers for net traffic.
If there are too many transmission errors the sending tcp drops the connection. My solution to this is to halve the size of CONFIG_SYS_RX_ETH_BUFFER until transmission works. If I was thinking about a buffer pool for in the drivers, I'd implement it in net/net.c. Interface "getbuffer," returns a pointer, and increments an index to the next net_rx_buffer with no protection for overrun. Overrun is managed by ack numbers in TCPand timeout in UDP.
Possibly CONFIG_SYS_RX_ETH_BUFFER could come under Kconfig.
Regards, Simon
RegardsDuncan
Hi Duncan,
On 22 April 2018 at 21:22, Duncan Hare dh@synoia.com wrote:
From: Simon Glass sjg@chromium.org To: Duncan Hare dh@synoia.com Sent: Sunday, April 22, 2018 1:10 PM Subject: Re: [PATCH v10 3/3] Adding wget
Hi Duncan,
On 17 April 2018 at 15:58, Duncan Hare dh@synoia.com wrote:
From: Simon Glass sjg@chromium.org
It has been through patman, and the only errors flagged a packed structures, necessary for packed headers.
It should be possible in the Python test to enable an http server on localhost with a few lines of code, and then connect to it in the test?
Yes server at port 80. The server can be tested with the wget command
which
can be installed on linux. I doubt that loop-back like this will produce the scrambling of packet
order
which is a feature of push down stacks for packet queues in the internet.
The pi is easy to overrun when testing on a low latency network, because
the
TCP send window size is defined by the number of net_rx_buffers, which is controlled the CONFIG_SYS_RX_ETH_BUFFER in include.net.h, which sets PKTBUFSRX at the beginning of include/net.h, which in-turn defines a buffer pool in net/net.c, array named
net_pkt_buf.
Hence my comment in a different thread about buffering on the pi. Few of
the
socs appear to use net_pkt_buf buffers for net traffic.
If there are too many transmission errors the sending tcp drops the connection. My solution to this is to halve the size of CONFIG_SYS_RX_ETH_BUFFER until transmission works.
If I was thinking about a buffer pool for in the drivers, I'd implement it in net/net.c. Interface "getbuffer," returns a pointer, and increments an index to the next net_rx_buffer with no protection for overrun. Overrun is managed by ack numbers in TCP and timeout in UDP.
Possibly CONFIG_SYS_RX_ETH_BUFFER could come under Kconfig.
Just to be clear, I was wondering about having an automated test. Manual tests are not very useful since people won't do them. See 'make tests' for all the test that we currently run. I'm pretty sure you could standard up a little server, run your wget, then shut it down, all within a pytest test.
Regards, Simon
From: Simon Glass sjg@chromium.org To: Duncan Hare dh@synoia.com Cc: U-Boot Mailing List u-boot@lists.denx.de; Joe Hershberger joe.hershberger@ni.com Sent: Tuesday, April 24, 2018 10:01 PM Subject: Re: [PATCH v10 3/3] Adding wget
Hi Duncan,
On 22 April 2018 at 21:22, Duncan Hare dh@synoia.com wrote:
The server can be tested with the wget command which can be installed on linux. I doubt that loop-back like this will produce the scrambling of packet order which is a feature of push down stacks for packet queues in the internet.
Hence my comment in a different thread about buffering on the pi. Few of the socs appear to use net_pkt_buf buffers for net traffic.
If there are too many transmission errors the sending tcp drops the connection. My solution to this is to halve the size of CONFIG_SYS_RX_ETH_BUFFER until transmission works.
>
Possibly CONFIG_SYS_RX_ETH_BUFFER could come under Kconfig.
Just to be clear, I was wondering about having an automated test. Manual tests are not very useful since people won't do them. See 'make tests' for all the test that we >currently >run. I'm pretty sure you could standard up a little server, run your wget, then shut it down, all within a pytest test.
Regards, Simon
Hi Wolfgang. Simon Can we put a test 4 Mbyte kernel on the u-boot website for an automated test for other users of TCP & Wget in u-boot? Then I can produce a standard u-boot script for testing. RegardsDuncan Hare
Hi Duncan,
On 25 April 2018 at 08:33, Duncan Hare dh@synoia.com wrote:
From: Simon Glass sjg@chromium.org To: Duncan Hare dh@synoia.com Cc: U-Boot Mailing List u-boot@lists.denx.de; Joe Hershberger joe.hershberger@ni.com Sent: Tuesday, April 24, 2018 10:01 PM Subject: Re: [PATCH v10 3/3] Adding wget
Hi Duncan,
On 22 April 2018 at 21:22, Duncan Hare dh@synoia.com wrote:
The server can be tested with the wget command which can be installed on linux. I doubt that loop-back like this will produce the scrambling of packet order which is a feature of push down stacks for packet queues in the internet.
Hence my comment in a different thread about buffering on the pi. Few of the socs appear to use net_pkt_buf buffers for net traffic.
If there are too many transmission errors the sending tcp drops the connection. My solution to this is to halve the size of CONFIG_SYS_RX_ETH_BUFFER until transmission works.
Possibly CONFIG_SYS_RX_ETH_BUFFER could come under Kconfig.
Just to be clear, I was wondering about having an automated test. Manual tests are not very useful since people won't do them. See 'make tests' for all the test that we >currently >run. I'm pretty sure you could standard up a little server, run your wget, then shut it down, all within a pytest test.
Regards, Simon
Hi Wolfgang. Simon
Can we put a test 4 Mbyte kernel on the u-boot website for an automated test for other users of TCP & Wget in u-boot?
Then I can produce a standard u-boot script for testing.
How about the test just creates a little (4KB) file. We don't want the tests to access a real network, if possible, just use localhost.
Regards, Simon
From: Simon Glass sjg@chromium.org To: Duncan Hare dh@synoia.com Cc: Wolfgang Denk wd@denx.de; U-Boot Mailing List u-boot@lists.denx.de; Joe Hershberger joe.hershberger@ni.com Sent: Wednesday, April 25, 2018 4:44 PM Subject: Re: [PATCH v10 3/3] Adding wget
Hi Duncan,
On 25 April 2018 at 08:33, Duncan Hare dh@synoia.com wrote: ____________________
From: Simon Glass sjg@chromium.org To: Duncan Hare dh@synoia.com Cc: U-Boot Mailing List u-boot@lists.denx.de; Joe Hershberger joe.hershberger@ni.com Sent: Tuesday, April 24, 2018 10:01 PM Subject: Re: [PATCH v10 3/3] Adding wget
Hi Duncan,
On 22 April 2018 at 21:22, Duncan Hare dh@synoia.com wrote:
The server can be tested with the wget command which can be installed on linux. I doubt that loop-back like this will produce the scrambling of packet order which is a feature of push down stacks for packet queues in the internet.
Hence my comment in a different thread about buffering on the pi. Few of the socs appear to use net_pkt_buf buffers for net traffic.
If there are too many transmission errors the sending tcp drops the connection. My solution to this is to halve the size of CONFIG_SYS_RX_ETH_BUFFER until transmission works.
Possibly CONFIG_SYS_RX_ETH_BUFFER could come under Kconfig.
Just to be clear, I was wondering about having an automated test. Manual
tests are not very useful since people won't do them. See 'make tests' for all the test that we >currently >run. I'm pretty sure you could standard up a little server, run your wget, then shut it down, all within a pytest test.
Regards, Simon
Hi Wolfgang. Simon
Can we put a test 4 Mbyte kernel on the u-boot website for an automated test for other users of TCP & Wget in u-boot?
Then I can produce a standard u-boot script for testing.
How about the test just creates a little (4KB) file. We don't want the tests to access a real network, if possible, just use localhost.
Regards, Simon
4k is 4 packets. I believe most kernels are larger. I was think of a static server set up with a known dns name. Thta's what I've got.
Do the test setup once.
On Wed, Apr 25, 2018 at 6:52 PM, Duncan Hare dh@synoia.com wrote:
From: Simon Glass sjg@chromium.org To: Duncan Hare dh@synoia.com Cc: Wolfgang Denk wd@denx.de; U-Boot Mailing List u-boot@lists.denx.de; Joe Hershberger joe.hershberger@ni.com Sent: Wednesday, April 25, 2018 4:44 PM Subject: Re: [PATCH v10 3/3] Adding wget
Hi Duncan,
On 25 April 2018 at 08:33, Duncan Hare dh@synoia.com wrote: ____________________
From: Simon Glass sjg@chromium.org To: Duncan Hare dh@synoia.com Cc: U-Boot Mailing List u-boot@lists.denx.de; Joe Hershberger joe.hershberger@ni.com Sent: Tuesday, April 24, 2018 10:01 PM Subject: Re: [PATCH v10 3/3] Adding wget
Hi Duncan,
On 22 April 2018 at 21:22, Duncan Hare dh@synoia.com wrote:
The server can be tested with the wget command which can be installed on linux. I doubt that loop-back like this will produce the scrambling of packet order which is a feature of push down stacks for packet queues in the internet.
Hence my comment in a different thread about buffering on the pi. Few of the socs appear to use net_pkt_buf buffers for net traffic.
If there are too many transmission errors the sending tcp drops the connection. My solution to this is to halve the size of CONFIG_SYS_RX_ETH_BUFFER until transmission works.
Possibly CONFIG_SYS_RX_ETH_BUFFER could come under Kconfig.
Just to be clear, I was wondering about having an automated test. Manual
tests are not very useful since people won't do them. See 'make tests' for all the test that we >currently >run. I'm pretty sure you could standard up a little server, run your wget, then shut it down, all within a pytest test.
Regards, Simon
Hi Wolfgang. Simon
Can we put a test 4 Mbyte kernel on the u-boot website for an automated test for other users of TCP & Wget in u-boot?
Then I can produce a standard u-boot script for testing.
How about the test just creates a little (4KB) file. We don't want the tests to access a real network, if possible, just use localhost.
This makes it portable to where ever the test is run.
Regards, Simon
4k is 4 packets. I believe most kernels are larger. I was think of a static server set up with a known dns name. Thta's what I've got.
Do the test setup once.
This assume accessibility to this Internet server and that this server is up /still configured when the test runs.
Please setup a test that can run in an environment without the Internet. That is critical for unit tests.
Hand tests for Internet usage and the environmental effects are great, but that can't be what we include in the auto tests for repeat-ability reasons. Simon is asking for a separate type of test.
U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Please setup a test that can run in an environment without the Internet. That is critical for unit tests.
Hand tests for Internet usage and the environmental effects are great, but that can't be what we include in the auto tests for repeat-ability reasons. Simon is asking for a separate type of test.
Is the test environment another patch in the series or an addition to the wget patch? Thanks Duncan
Hi,
On 14 May 2018 at 07:05, Duncan Hare dh@synoia.com wrote:
Please setup a test that can run in an environment without the Internet. That is critical for unit tests.
Hand tests for Internet usage and the environmental effects are great, but that can't be what we include in the auto tests for repeat-ability reasons. Simon is asking for a separate type of test.
Is the test environment another patch in the series or an addition to the wget patch?
I suggest a separate patch.
Regards, Simon
From: Simon Glass sjg@chromium.org
To: Duncan Hare dh@synoia.com Cc: "joe.hershberger@ni.com" joe.hershberger@ni.com; U-Boot Mailing List u-boot@lists.denx.de Sent: Sunday, May 13, 2018 3:00 PM Subject: Re: net: [U-Boot] [PATCH v10 3/3] Adding wget
Please setup a test that can run in an environment without the Internet. That is critical for unit tests.
Hand tests for Internet usage and the environmental effects are great, but that can't be what we include in the auto tests for repeat-ability reasons. Simon is asking for a separate type of test.
Is the test environment another patch in the series or an addition to the wget patch?
I suggest a separate patch.
Regards,>>Simon
Separate patch, patch 4 of the series or a completely new patch? Conceptual approach:
apt-get install nginx (in debian) I'll provide a config file pointing to test kernel a a specified file location.
Test kernel will be numbered lines of printable characters. Printable, not binary, easy to expand in a spreadsheet.
Can the u-boot print command print sections of memory? That is print $loadaddr $length?
That will work for small test kernels, or
I tested by using tftp to download a kernel, then used a modified wget to download a seconf time and verify thedownloads were identical.
compare $loadaddr1 $loadaddr2 $length
Is that a preferred approach? Duncan Hare
714 931 7952
Hi Duncan,
On 14 May 2018 at 09:26, Duncan Hare dh@synoia.com wrote:
From: Simon Glass sjg@chromium.org
To: Duncan Hare dh@synoia.com Cc: "joe.hershberger@ni.com" joe.hershberger@ni.com; U-Boot Mailing List u-boot@lists.denx.de Sent: Sunday, May 13, 2018 3:00 PM Subject: Re: net: [U-Boot] [PATCH v10 3/3] Adding wget
Please setup a test that can run in an environment without the Internet. That is critical for unit tests.
Hand tests for Internet usage and the environmental effects are great, but that can't be what we include in the auto tests for repeat-ability reasons. Simon is asking for a separate type of test.
Is the test environment another patch in the series or an addition to the wget patch?
I suggest a separate patch.
Regards,
Simon
Separate patch, patch 4 of the series or a completely new patch?
Patch 4 in the series.
Conceptual approach:
apt-get install nginx (in debian)
I'll provide a config file pointing to test kernel a a specified file location.
Test kernel will be numbered lines of printable characters. Printable, not binary, easy to expand in a spreadsheet.
Can the u-boot print command print sections of memory? That is
print $loadaddr $length?
That will work for small test kernels, or
I tested by using tftp to download a kernel, then used a modified wget to download a seconf time and verify the downloads were identical.
compare $loadaddr1 $loadaddr2 $length
Is that a preferred approach?
You don't need to download a kernel. Just download a small file (say 10KB) that you generate in your test. You might find test_vboot.py and test_net.py helpful. Something like:
- start an http server on localhost (in Python) - put a single file in the server that can be read (generate 10KB of known data?) - run U-Boot with the commands to wget from that server - check that U-Boot gets the expected file
Regards, Simon
participants (3)
-
Duncan Hare -
Joe Hershberger -
Simon Glass