Bug 1124509

Summary: Screen corruption using serial-getty@ttyS0
Product: [Fedora] Fedora Reporter: David Jones <djones-proj>
Component: minicomAssignee: Jaromír Cápík <jcapik>
Status: CLOSED WORKSFORME QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 21CC: djones-proj, jcapik, ovasik
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-06-16 21:02:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Show ls listing for / /boot /bin none

Description David Jones 2014-07-29 16:23:50 UTC
Created attachment 922203 [details]
Show ls listing for / /boot /bin

Description of problem:
Using a pl2303 USB Serial convertor to a Cubietruck as soon as the kernel starts to load from extlinux.conf there is screen corrution. 
This has been a problem with both Fedora 21 and Rawhide, since at least the Fedora-Minimal-armhfp-rawhide from the 13/07/2014.
The lastest image being tested is Fedora-Minimal-armhfp-21-20140729-sda.raw.xz.


Version-Release number of selected component (if applicable):
minicom version 2.6.2

How reproducible:
As soon as the kernel starts loading.

Steps to Reproduce:
1. Poweroff the Cubietruck
2. Restart the Cubietruck no screen corruption until the after kernel is selected in the /boot/extlinux/extlinux.conf and starts to load.
3.

Actual results:
See attachment.

Expected results:


Additional info:
Lines added from dmesg showing initialization of serial port on Cubietruck.
[    5.299684] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    5.304398] console [ttyS0] disabled
[    5.324679] 1c28000.serial: ttyS0 at MMIO 0x1c28000 (irq = 33, base_baud = 1500000) is a U6_16550A
[    6.145946] console [ttyS0] enabled

I read somewhere minicom is unreliable, I have also tried:-
Screen version 4.01.00devel
GTKTerm version 0.99.7-rc1
With the same screen corruption.

All software tried is configured baud 115200 parity N bits 8 stopbits 1

I use the same pl2303 USB Serial convertor and minicom with no problems for connecting to 2 TPlink routers running Openwrt, cubieboard2, Cubietruck with remix of Fedora 19/20 from Hans de goede based on sunxi 3.4.75

Comment 1 Jaromír Cápík 2014-08-05 19:37:42 UTC
Hello David.

This really doesn't look like a minicom problem. It looks more like two possible issues ... depending on their reproducibility.

1.) If you get different mess with each listing, it could be caused by a noise on the Cubietruck TX line making the pl2303 receiver to get out of sync and as there are more data in line, some of the hi-low transitions are probably interpreted as consequent stop/start bit pairs.

2.) If you get the same mess with each try, maybe your filesystem is corrupted and you simply print content of some binary file.

Could you please check the filesystem?

Thanks,
Jaromir.

Comment 2 David Jones 2014-08-05 22:04:31 UTC
(In reply to Jaromír Cápík from comment #1)
> Hello David.
> 
> This really doesn't look like a minicom problem. It looks more like two
> possible issues ... depending on their reproducibility.
> 
> 1.) If you get different mess with each listing, it could be caused by a
> noise on the Cubietruck TX line making the pl2303 receiver to get out of
> sync and as there are more data in line, some of the hi-low transitions are
> probably interpreted as consequent stop/start bit pairs.
> 
> 2.) If you get the same mess with each try, maybe your filesystem is
> corrupted and you simply print content of some binary file.
> 
> Could you please check the filesystem?
> 
> Thanks,
> Jaromir.

Hello jaromir

I do not believe this to be a filesystem corruption. I am following the F21,  Rawhide releases and have a number of different images installed on SD cards, these are either:-

Installed from koji images and follow a number of daily updates

or

Fresh koji images installed.

As stated before I use this pl2303 for serial connection to:- 
2 TP WR710N routers with Openwrt installed. 
1 cubieboard2 with Fedora 20 remix (Hans de goede) based on sunxi 3.4.75 kernel. 
1 cubietruck with Fedora 20 remix (Hans de goede) based on sunxi 3.4.75 kernel.

All of these systems serial connection work perfectly well.

The cubietruck I am using for testing F21 and Rawhide is the only system with this corruption.

Example of corruption while logging in to system running F22 (Rawhide)

Welcome to minicom 2.6.2

OPTIONS: I18n 
Compiled on Aug  7 2013, 13:32:48.
Port /dev/ttyUSB0, 15:05:56

Press CTRL-A Z for help on special keys

Password: 
Login incrrgct

ct1 login� root
Password: 
Last login: Tue�Aug  ������҂

Comment 3 Jaromír Cápík 2014-08-06 11:09:14 UTC
In that case it must be a hardware issue (broken oscillator, dead TxD pin on the transmitter - unable to source enough current to drive the output voltage levels correctly, broken cable, etc.). Do you have a digital oscilloscope or at least a multimeter? I would check the TxD pin output voltage on the affected Cubieboard. Also check the serial cable - especially the GND wire conductivity. Sometimes the GND wire is broken and consequently the ground voltage is undefined and the data signals are missing a voltage reference. In that case any voltage spikes or changes in the power consumption can cause unwanted transitions on the data wires. It's really very unlikely caused by minicom. Especially when you get the same mess even with screen and GTKTerm.

Comment 4 Jaromír Cápík 2014-08-13 17:39:52 UTC
any news here?

Comment 5 David Jones 2014-08-13 23:09:16 UTC
(In reply to Jaromír Cápík from comment #4)
> any news here?

Hello jaromir

Sorry for delay in replying, I was away for a long weekend with family.

I have tested the PL2303 serial device again with all equipment listed previously.
No problems except when using with Fedora 21 and 22 (Rawhide).

Since performing upgrades on Fedora 22 there appears to be some improvement over the corruption, as you can see below. Note everytime I boot the F21 and F22 systems there is no corruption until the kernel starts loading.

I do not think this is a problem with minicom. Could it be a timing issue in the Kernels for F21 and F22. Do you know some one in kernel development that could take a look at this.

U-Boot SPL 2014.04 (Jun 08 2014 - 07:10:42)
DRAM: 2048 MiB


U-Boot 2014.04 (Jun 08 2014 - 07:10:42) Allwinner Technology

CPU:   Allwinner A20 (SUN7I)
DRAM:  2 GiB
MMC:   SUNXI SD/MMC: 0
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Hit any key to stop autoboot:  0 
MMC Device 2 not found
no mmc device at slot 2
MMC Device 1 not found
no mmc device at slot 1
mmc0 is current device
Scanning mmc 0...
Found extlinux config /extlinux/extlinux.conf
Retrieving file: /extlinux/extlinux.conf
1164 bytes read in 101 ms (10.7 KiB/s)
Ignoring unknown command:  menu.c32
Ignoring unknown command:  600
Ignoring unknown command: default=Fedora
Fedora-Minimal-armhfp-rawhide-20140722 Boot Options.
1:      Fedora (3.16.0-1.fc22.armv7hl) 22 (Rawhide)
2:      Fedora (3.16.0-0.rc7.git4.1.fc22.armv7hl) 22 (Rawhide)
3:      Fedora (3.16.0-0.rc7.git1.1.fc22.armv7hl) 22 (Rawhide)
Enter choice: 1:        Fedora (3.16.0-1.fc22.armv7hl) 22 (Rawhide)
Retrieving file: /initramfs-3.16.0-1.fc22.armv7hl.img
11913817 bytes read in 673 ms (16.9 MiB/s)
Retrieving file: /vmlinuz-3.16.0-1.fc22.armv7hl
5455824 bytes read in 690 ms (7.5 MiB/s)
no console= 
append: ro root=UUID=d2b34015-1dc1-4043-868c-470944871978 LANG=en_US.UTF-8 console=ttyS0,115200
Retrieving file: /dtb-3.16.0-1.fc22.armv7hl/sun7i-a20-cubietruck.dtb
21639 bytes read in 1297 ms (15.6 KiB/s)
Kernel image @ 0x42000000 [ 0x000000 - 0x533fd0 ]
## Flattened Device Tree blob at 50000000
   Booting using the fdt blob at 0x50000000
   Loading Ramdisk to 4f4a3000, end 4ffffa59 ... OK
   Loading Device Tree to 4f49a000, end 4f4a2486 ... OK

Starting kernel ...

    0>00000�] Bo�ing Lo�� � ph{s��� hw���
�   �.0000��_ Y�o�<+[��g c����subs{s 6��w������r���0w/����|���������s{��6�[����r�����m�z�,��?���������>���?
                                                                                                                  /���|�����}���������������3^�����������?���|����������������?���������������������������������k
[    0.009095] ftrace: allocating 28931 entries in 57 pages
[    0.094022] /cpus/cpu@0 missing clock-frequency property
[    0.094068] /cpus/cpu@1 missing clock-frequency property
[    0.094095] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[    0.094244] Setting up static identity map for 0x408720d8 - 0x40872170
[    0.115274] CPU1: Booted secondary processor
[    0.115354] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
[    0.115566] Brought up 2 CPUs
[    0.115606] SMP: Total of 2 processors activated.
[    0.115620] CPU: All CPU(s) started in HYP mode.
[    0.115633] CPU: Virtualization extensions available.
[    0.117475] devtmpfs: initialized
[    0.127406] VFP support v0.3: implementor 41 architecture 2 part 30 variant 7 rev 4
[    0.142590] atomic64_test: passed
[    0.142813] pinctrl core: initialized pinctrl subsystem
[    0.144038] regulator-dummy: no parameters
[    0.160212] NET: Registered protocol family 16
[    0.165724] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.169538] cpuidle: using governor menu
[    0.186164] No ATAGs?
[    0.186251] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[    0.186274] hw-breakpoint: maximum watchpoint size is 8 bytes.
[    0.186618] EXYNOS: PMU not supported
[    0.189440_ Serial: AMBC PL011 UART driver
[    0.220855] edoa-�mc-enging e�ma/d��engine.0: Can't alloca�e PcRAM dwm}_����7
                                                                                {�   0.22093;}�edmc-dma/gn�˫��˽�g � ed}�KVk�e�ging.� fakle�`wk�� error�-7                                                       l
[    0.221929] reg-fixed-voltage usb1-vbus: could not find pctldev for node /soc@01c00000/pinctrl@01c20800/usb1_vbus_pin@0, deferring probe                 [    �.22n�������ɥᗳ�volt�V�����uv: ���d ���fi�d p   �
[    0.221961] platform usb1-vbus: Driver reg-fixed-voltage requests probe deferral
[    0.222012] reg-fixed-voltage usb2-vbus: could not find pctldev for node /soc@01c00000/pinctrl@01c20800/usb2_vbus_pin@0, deferring probe
[    0.222043] platform usb2-vbus: Driver reg-fixed-voltage requests probe deferral
[    0.222519] vcc3v0: 3000 mV 
[    0.223158] vcc3v3: 3300 mV 
[    0.223402] reg-fixed-voltage vmmc3: could not find pctldev for node /soc@01c00000/pinctrl@01c20800/vmmc3_pin@0, deferring probe
[    0.223438] platform vmmc3: Driver reg-fixed-voltage requests probe deferral
[    0.227072] vgaarb: loaded
[    0.229863] SCSI subsystem initialized
[    0.231214] usbcore: registered new interface driver usbfs
[    0.231389] usbcore: registered new interface driver hub
[    0.231711] usbcore: registered new device driver usb
[    0.234348] exynos_iommu_init: Failed to register exynos-iommu driver.
[    0.236138] NetLabel: Initializing
[    0.236160] NetLabel:  domain hash size = 128
[    0.236174] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.236299] NetLabel:  unlabeled traffic allowed by default
[    0.237357] Switched to clocksource arch_sys_cownter
[    0.383354] NET: Registered protocol family 2
                                                 [    0.387239] TCP establksxed hcsh tcbl6*��ɥ��'ʲBz˱�.'�b�����2�ѕϧ
[    �.387461_ t42����hcsx�table�entry�� 8192 (order: 4, 65536 bytes)
[    0.385760] TCP: Hash tables configured (established 8192 bind 8192)
[    0.385911] TCP: reno registered
[    0.385945] UDP hash table entries: 512 (order: 2, 16384 bytes)
[    0.386037] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[    0.386604] NET: Registered protocol family 1
[    0.387550] RPC: Registered named UNIX socket transport module.
[    0.387576] RPC: Registered udp transport module.
[    0.387592] RPC: Registered tcp transport module.
[    0.387607] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.388616] Unpacking initramfs...
[    2.305353] Freeing initrd memory: 11632K (cf4a3000 - cffff000)
[    2.312874] hw perfevents: enabled with ARMv7 Cortex-A7 PMU driver, 5 counters available
[    2.317091] futex hash table entries: 512 (order: 3, 32768 bytes)
[    2.317439] audit: initializing netlink subsys (disabled)
[    2.317551] audit: type=2000 audit(2.280:1): initialized
[    2.347816] zbud: loaded
[    2.349927] VFS: Disk quotas dquot_6.5.2
[    2.350518] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    2.356524] NFS: Registering the id_resolver key type
[    2.356626] Key type id_resolver registered
[    2.356644] Key type id_legacy registered
[    2.356697] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
{    2.357558] osgmni has begn set to 1453
                                           [    2.358466_ Key type big_ke{ r�Vkїɕ�5
                                   �������3�                                         [    2.366300] a|g: No �est fˁstdr�g ({�ng)                                                                                3
[    3.326021] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel� 2}6������ ����LW����˃�ð���?��C��>�������������������>����������.�����{�������-��������������                �
[    3.336181] hidraw: raw HID events driver (C) Jiri Kosina
[    3.343931] usbcore: registered new interface driver usbhid

Comment 6 Jaromír Cápík 2014-08-14 11:01:19 UTC
Hello David.

Does it mean you now experience the same issue on multiple boards?
Sorry, but this really looks like a broken cable + race condition + coincidence during your tests. Feel free to switch the component to kernel, but I don't believe it is caused by the kernel. The serial timing is usually generated on the HW level and the only kernel role is to switch between the baud rates. That would have to be a memory error in the kernel, causing the serial buffers to be corrupted or something. My second suspicion was that two processes are trying to write to the same serial device for some reason.

Regards,
Jaromir.

Comment 7 David Jones 2014-08-18 11:28:35 UTC
(In reply to Jaromír Cápík from comment #6)
> Hello David.
> 
> Does it mean you now experience the same issue on multiple boards?
> Sorry, but this really looks like a broken cable + race condition +
> coincidence during your tests. Feel free to switch the component to kernel,
> but I don't believe it is caused by the kernel. The serial timing is usually
> generated on the HW level and the only kernel role is to switch between the
> baud rates. That would have to be a memory error in the kernel, causing the
> serial buffers to be corrupted or something. My second suspicion was that
> two processes are trying to write to the same serial device for some reason.
> 
> Regards,
> Jaromir.

Hello Jaromir
Thanks very much for all your help. This problem may become more noticeable by larger numbers of users as they start to adopt Fedora 21 after it's release.
 
A little more background, I use the same laptop with Fedora 20 to connect by USB serial. All of the following work; 
Fedora Allwinner Cubietruck and Cubieboard 2 remix 3.4.75 by Hans de Goede. Android on the Cubietruck and Cubieboard. 
Openwrt different versions on two TL-WR710N routers. 

I only have this problem with Fedora 21 and Fedora rawhide. The kernel command line for the latest version of rawhide is:

cat /boot/extlinux/extlinux.conf
# extlinux.conf generated by appliance-creator
ui menu.c32
menu autoboot Welcome to Fedora-Minimal-armhfp-rawhide-20140813. Automatic boot in # second{,s}. Press a key for options.
menu title Fedora-Minimal-armhfp-rawhide-20140813 Boot Options.
menu hidden
timeout 20
totaltimeout 600

default=Fedora-Minimal-armhfp-rawhide-20140813 (3.17.0-0.rc0.git4.2.fc22.armv7hl)
label Fedora (3.17.0-0.rc0.git7.1.fc22.armv7hl) 22 (Rawhide)
        kernel /vmlinuz-3.17.0-0.rc0.git7.1.fc22.armv7hl
        append ro root=UUID=880a25a2-5c62-454a-84dd-61de724d7551 LANG=en_US.UTF-8
        fdtdir /dtb-3.17.0-0.rc0.git7.1.fc22.armv7hl/
        initrd /initramfs-3.17.0-0.rc0.git7.1.fc22.armv7hl.img

a) If you notice from my previous reply there is no corruption between the lines 
U-Boot SPL 2014.04 (Jun 08 2014 - 07:10:42) and Starting kernel ...

The corruption only starts after Uboot hands over to linux loading the kernel.

b) I have tried adding console=ttyUSB0,115200n8 after LANG=en_US.UTF-8 on the line starting with append. This does not appear to change the output when the kernel starts

c) Dmesg output for the serial connection
[    5.070591] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    5.106439] console [ttyS0] disabled
[    5.128764] 1c28000.serial: ttyS0 at MMIO 0x1c28000 (irq = 33, base_baud = 1500000) is a U6_16550A
[    6.048597] console [ttyS0] enabled
[    6.057594] Serial: AMBA driver
[    6.062337] Serial: IMX driver
[    6.068066] msm_serial: driver initialized

Could the problem be either

1) The base_baud setting shown above is incorrect.

or

2) A conflict between the Uboot serial device and the kernel serial device?