Helios, a FireWire stack for MorphOS
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    Stevo
    Posts: 797 from 2004/1/24
    From: #AmigaZeux
    Kewl, can't wait :-)
    ---
    http://www.iki.fi/sintonen/logs/its_only_football.txt
  • »26.01.09 - 16:23
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    ok, it's done.

    For some hours I've put the archive on my personal 'Free' ftp account:

    http://www.yomgui.fr/projects/Helios_0.1.162.lha

    But I've uploaded it on Aminet also, we need to wait the validation.
    Please read the README (and with multiview! because it's an UTF-8 encoding text :-O)
    And now... next project!
  • »26.01.09 - 20:45
    Profile Visit Website
  • rms
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    rms
    Posts: 577 from 2004/11/27
    FANTASTIC :-)

    Really great, I'm looking forward to test it.

    Regards

    Christoph
  • »27.01.09 - 06:48
    Profile Visit Website
  • jPV
  • Paladin of the Pegasos
    Paladin of the Pegasos
    jPV
    Posts: 1346 from 2003/2/24
    From: po-RNO
    Quote:


    SoundSquare wrote:
    Quote:

    That's what sbp2 is for, also usb2 pci cards are cheap and easily available.


    indeed, but USB2 in Poseidon sucks (not related to Poseidon directly i think but to the driver). It's unstable, often slow, doesn't work with every usb2 devices, and easily crash with large transfers.



    Hmmm.. or maybe it's about the setup :). USB2 PCI card has worked fine on my Peg1. I've filled and refilled my 250G USB-HD without any problems. With large transfers like movies, music and cd images. Also handfull of USB pens have worked. And couple of cameras. And HIDs like several mice and driving wheel... only problem was with bad cable which came with HD, I had drops with it, but after replacing it with proper cable everything has worked perfectly.

    But anyway firewire is welcome. Sounds much better by design and USB always has some kludge feeling, I don't know why ;)

    [ Edited by jPV on 2009/1/27 19:57 ]
  • »27.01.09 - 17:54
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Posts: 408 from 2004/7/15
    From: Russia, Moscow
    Is it going to be opensourced ?
    I'm asking because i still feel ashamed for my complete failure. :-( I don't have time to develop its core, but may be i could at least help with something?
    I could test device discovery and could help to improve GUI. I think i can't do anything more because my video camera is broken and it won't output correct video data.
    iPod, iBook, iMac,... iRobot?
  • »12.02.09 - 14:05
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    New release done this morning on Aminet:

    http://aminet.net/driver/other/Helios_0.3.323.lha

    This one handle isochronous mode in receive mode.
    So I've made a small code example to show how to use that.

    This example is useful also because it permit to capture the broadcast audio/video stream from a camcorder supporting the IEC 61883 protocol and giving MPEG2-TS or DV format.

    the code is located in SDK/Examples/, file get61883.c.
    Compile it using command 'make get61883' in its directory if you've installed the MorphOS SDK.

    For thus they don't want or don't able to build it, this is a binary:

    http://www.yomgui.fr/projects/get61883

    First thing to do before launching it is to execute FWInspect before to let it enumerate devices and their capabilities. then get61883

    Simple usage: get61883 [-dv] output

    -dv is optional and switch into DV format (MPEG2-TS by default).
    output is the output file where the stream will be written.

    CTRL-E to finish the write at any moment.
    The file is written using the asyncio library.

    Good experimentation ;-)

    [ Edited by Yomgui on 2009/3/27 7:09 ]

    [ Edited by Yomgui on 2009/3/27 7:18 ]
    And now... next project!
  • »27.03.09 - 10:05
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    I've just uploaded the new Helios on Aminet.
    It has been completly re-written to have a design and API more close to the one made for Poseidon.

    So any previous installation must be removed before!!

    In this version, no GUI application yet :-(
    Just one command to run: helios_rom_start and nothing else.

    Helios and 3rd parties ouput some debug into the serial console, if you want to see bus enumeration and SBP2 info (SBP2 = protocol for HD, CDROM, Printers, ... like a super mass storage).

    The archive has just been put on Aminet so wait validation from their side.

    Archive name: driver/other/Helios_0.4-svn_r560.lha

    Ah forgotten to say: yes, SBP2.class is inside (see installation in README). So you can connect external FW harddrive enclosure and CD-ROM also.
    Partition and CDROM are automatically mounted after successfull enumeration by the sbp2.class (sbp2.device is created and added to the system at runtime).

    Use Mount and UnitControl MorphOS applications to play with partitions and CDROM mountage if the auto-mount doesn't work.

    Sorry the documentation is really short but my real life eats everything ;-)

    [ Edited by Yomgui on 2010/9/25 15:50 ]
    And now... next project!
  • »25.09.10 - 13:46
    Profile Visit Website
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    boot_wb
    Posts: 874 from 2007/4/9
    From: Kingston upon ...
    Holy shit, that's fantastic!

    [/rushes off to Ebay to but firewire enclosure]

    Thankyou for taking the time to give us a release: it was good to see firewire hard drive support in the pictures from the recent show (I forget which one it was...), so to have this available to users is great. :pint:
    www.hullchimneyservices.co.uk

    UI: Powerbook 5,6 (1.67GHz, 128MB VRam): OS3.1, OSX 10.5.8
    HTPC: Mac Mini G4 (1,5GHz, 64MB VRam): OS3.1 (ZVNC)
    Audiophile: Efika 5200b (SB Audigy): OS3.1 (VNC + Virtual Monitor)

    Windows free since 2011!
  • »25.09.10 - 14:10
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    As this revision is now "stable" enough to play with, I think it's time to release it. I need plugfest reports now.

    Knows bugs are:

    - Oxford ATA bridges (911/935 families) are known to no work after 2339 ORBs send from the computer. After a bus reset is needed to continue.
    - Multiple SBP2 devices on the same bus has not been really tested.
    - Others kind of devices are not supported yet (more .class needed ;-))
    - Code efficiency is not the first goal, so don't be surprised to see USB speed higer than FW...! Specially when the client software doesn't request data enough per IO command.

    [ Edited by Yomgui on 2010/9/25 17:15 ]
    And now... next project!
  • »25.09.10 - 15:08
    Profile Visit Website
  • Acolyte of the Butterfly
    Acolyte of the Butterfly
    vulture
    Posts: 138 from 2008/2/4
    From: Greece
    Awesome! Can't thank you enough!
  • »25.09.10 - 19:09
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    I'm interested in known helios behaviour with FW iPod generations.
    If someone owns this kind of devices, please to record the LogTool output and send me the result on my gmail.com account: yomgui1

    Thanks!
    And now... next project!
  • »26.09.10 - 09:40
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Divinity
    Posts: 462 from 2009/9/8
    @Yomgui
    It's FANTASTIC !!! :-)
    Connected with Pegasos2 G4 in port1 firewire a HD FAT32, in port 2 firewire a HD HFS+ (OS Leopard 10.5.x) and then connected in port 2 a notebook Macbook pro x86 in target mode firewire ! with OSX Snow Leopard 10.6.4 (HFS+) and Bootcamp with Windows (NTFS) (using an adaptor firewire 800 to firewire 400), ALL OK !!! fantastic ! :-)

    great work Yomgui !

    [ Edited by Divinity on 2010/9/26 11:48 ]
  • »26.09.10 - 10:46
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    Release reports are comming and it has started to come to fruition!
    I've fixed a major bug blocking to use Helios on Powermac.
    Thanks to analogkid to have reported the pb.

    I'll just delay a bit the release as I'm trying to introduce isochronous support.
    And now... next project!
  • »28.09.10 - 10:58
    Profile Visit Website
  • Paladin of the Pegasos
    Paladin of the Pegasos
    jcmarcos
    Posts: 1178 from 2003/3/13
    From: Pinto, Madrid ...
    Quote:

    Yomgui wrote:

    I'm trying to introduce isochronous support.


    Brilliant! Was that isocronous thingie the bit missing in some Poseidon drivers, preventing the use of some interesting devices?
  • »28.09.10 - 13:34
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    @jcmarcos: for sure it's missing. hopefully most of USB devices seem to be limited to the bulk mode.
    And now... next project!
  • »28.09.10 - 14:31
    Profile Visit Website
  • Paladin of the Pegasos
    Paladin of the Pegasos
    jcmarcos
    Posts: 1178 from 2003/3/13
    From: Pinto, Madrid ...
    Quote:

    Yomgui wrote:

    most of USB devices seem to be limited to the bulk mode.


    Aha. I was recalling that thread about having isochronous transfers in the USB stack, in order to use things like USB audio cards.
  • »28.09.10 - 15:35
    Profile
  • MorphOS Developer
    cyfm
    Posts: 369 from 2003/4/11
    From: Germany
    Don't mix firewire and USB isochronous transfers. It seems you have no clue what you are talking about ....
    Isochrounous Firewire transfers are at least totally unrelated to Poseidon/USB
  • »28.09.10 - 15:46
    Profile Visit Website
  • Yokemate of Keyboards
    Yokemate of Keyboards
    magnetic
    Posts: 2129 from 2003/3/1
    From: Los Angeles
    Yomgui!

    You are an Uber Hero! I'm going to try to hook up my G-Raid 512gb FW 800 drive to my peg2 when i get back to town in a few days. I kind of half ass read the thread.. can I do reads and writes at good speeds? This will be great to backup my peg2 morphos drives.. I Also have a Lacie FW dvdr drive wonder if that will work?
    Pegasos 2 Rev 2B3 w/ Freescale 7447 "G4" @ 1ghz / 1gb Nanya Ram
    Quad Boot: MorphOS 2.7 | Amiga OS4.1 U4 | Ubuntu PPC GNU/Linux | OS X 10.4
  • »28.09.10 - 22:32
    Profile Visit Website
  • Paladin of the Pegasos
    Paladin of the Pegasos
    jcmarcos
    Posts: 1178 from 2003/3/13
    From: Pinto, Madrid ...
    Quote:

    pega-1 wrote:

    Don't mix firewire and USB isochronous transfers. It seems you have no clue what you are talking about... Isochrounous Firewire transfers are at least totally unrelated to Poseidon/USB


    I know. I was not mixing, but putting side to side. It would be nice, but strange, having this function in the much less common IEEE 1394 interface.
  • »29.09.10 - 08:19
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    @magnetic:

    I've got also a LaCie DVD writer, maybe it's the same one: in this case Yes, it works! :-)

    So yup: sbp2.class has full read/write support. (so becarefull also!! data lost may happen... helios is quite young!)

    About speed: sadly, I've not reach good performance here with my usb/fw enclosure. USB is faster by 1 or 2 MB/s... it's not lot and I thing I've to check for what, it could be solved in a near futur (2 more weeks? :-))

    @pega-1: yep we're talking about the function, not the API/design.

    [ Edited by Yomgui on 2010/9/29 10:37 ]
    And now... next project!
  • »29.09.10 - 08:34
    Profile Visit Website
  • MorphOS Developer
    cyfm
    Posts: 369 from 2003/4/11
    From: Germany
    Quote:


    jcmarcos schrieb:
    Quote:

    pega-1 wrote:

    Don't mix firewire and USB isochronous transfers. It seems you have no clue what you are talking about... Isochrounous Firewire transfers are at least totally unrelated to Poseidon/USB


    I know. I was not mixing, but putting side to side. It would be nice, but strange, having this function in the much less common IEEE 1394 interface.


    Actually, USB iso transfers were implemented before we switched to the unified pciusb drivers. Some of the seperate UHCI/OHCI/EHCI drivers (at least the Pegasos UHCI) had it.
    I couldn't care less to add it to pciusb , though ....
  • »29.09.10 - 17:11
    Profile Visit Website
  • Yokemate of Keyboards
    Yokemate of Keyboards
    magnetic
    Posts: 2129 from 2003/3/1
    From: Los Angeles
    Yomgui

    Ah allright excellent news! Good job. the data loss scares me though! :)

    About the dvd drive what model number is yours? is it grey with a black drive?

    As far as speed goes shouldnt the fw 400 port on the motherboard be faster than any usb2 setup when optimized?
    Pegasos 2 Rev 2B3 w/ Freescale 7447 "G4" @ 1ghz / 1gb Nanya Ram
    Quad Boot: MorphOS 2.7 | Amiga OS4.1 U4 | Ubuntu PPC GNU/Linux | OS X 10.4
  • »29.09.10 - 20:32
    Profile Visit Website
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 9611 from 2003/5/22
    From: Germany
    > As far as speed goes shouldnt the fw 400 port on the motherboard be faster
    > than any usb2 setup when optimized?

    The theoretical maximum transfer rates are 400 Mbit/s for FW400 and 480 Mbit/s for USB2.
  • »29.09.10 - 20:50
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    Well, final speed depends on many parameters, and not only physicals ones.

    For examples, do a copy of a big files (likes more 100MB), using ambient and an hd enclosure with both connectic: USB2.0/FW400.

    Do it from the HD to the RAM (a Read so) and do it again in the opposite way (a Write).

    Then do the same process but use the shell command 'Copy' with parameter 'buf=2048'.

    In this process with have not changed any physicals parameters, only a software one: the amount of data to transport per SCSI command.

    USB and FW uses stricly the same SCSI command to operate read/write actions.

    Now in USB data comming from the USB bridge are copied by the CPU at the place requested by the application.
    In FW, incoming data are copied into this memory place by a DMA unit.

    So now think about this: the CPU copy is slow specially when data are not "well presented" (bad alignements, not in cache, ...), and indeed the CPU is busy to this task.
    The DMA copy is fast because the DMA is dedicated to this copy task.
    At the end of the copy we only have to indicate to the CPU data cache the full area to mark as trashed to be sure that a cached read is sync.

    But there is a pb with the DMA: it needs to be programmed. And this program is a block of memory prepared by the CPU.
    The USB doesn't have such software overhead.

    So you see that the USB may be faster if the FW stack is too slow to program the DMA and if requested data transfer is also too small to benefit of the DMA copy.

    Actually, I've not given many time to see how fast and efficient is my Helios DMA programming code.
    So if we're in the cross point where block is too small, the USB 2.0 transfer may be as fast or faster than FW.

    That's why I request you to use the buf= parameter of Copy to increase this per command buffer size.

    And finally, I've also seen on internet that some FW bridge are badly designed and are slower than the USB one to communicate to the ATA bridge, the one that effectively drives the HD.

    For information: with my setup, I've reached a max of 25 MegaByte/s on a read transfer to RAM: using copy and buf=2048 (that gives 1MB buffer as buf uses HD block size unit = 512 bytes in most cases).
    Someone else, using a FW800 plug, but connected to a FW400 hd enclosure (anyway Helios limits to S400 the max speed, as it doesn't support higher speeds configurations) has reach the speed of 40MB/s!

    You see speed is an highly undefined thing.

    Note2:
    An SBP2 transfer of data is defined by sending an ORB command. This command transport the SCSI cmd data and a pointer to a scatter-gather (SG) table to define where wanted data need to be put by the DMA.
    Each entry of the SG table limits the size of described memory area by the number of bits to indicate this block size: 16bits here, so a theorical maximum of 65535 bytes. But for DMA performance and alignement compliance, I've limited it to 65532 bytes (to be aligned on 4 bytes).

    So using a 1MB data transfer needs filling: 1*1024*1024/65532 ~= 17 entries. Each entry is 8 bytes long => 136 bytes of write overhead compared to the USB.

    [ Edited by Yomgui on 2010/9/30 11:15 ]

    [ Edited by Yomgui on 2010/9/30 11:18 ]
    And now... next project!
  • »30.09.10 - 08:54
    Profile Visit Website
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 9611 from 2003/5/22
    From: Germany
    >> The theoretical maximum transfer rates are 400 Mbit/s for FW400 and
    >> 480 Mbit/s for USB2.

    > final speed depends on many parameters, and not only physicals ones. [...]

    Thanks for your detailed explanation. Wikipedia confirms what you said:

    "Although current high-speed USB 2.0 (introduced in 2001) is quoted as running at a higher signaling rate (480 Mbit/s) than legacy FireWire 400 (400 Mbit/s, available since 1995), data transfers over S400 FireWire interfaces generally outperform similar transfers over USB 2.0 interfaces. Few if any USB 2.0 device implementations are able to saturate the entire 480 Mbit/s, but this can be achieved with multiple devices on the same bus. Typical USB PC hosts rarely can sustain transfers exceeding 280 Mbit/s, with 240 Mbit/s being more typical. This is likely due to USB's reliance on the host processor to manage low-level USB protocol, whereas FireWire delegates the same tasks to the interface hardware (requiring less or no CPU usage). For example, the FireWire host interface supports memory-mapped devices, allowing high-level protocols to run without loading the host CPU with interrupts and buffer-copy operations. Besides throughput, other differences are that FireWire uses simpler bus networking, provides more power over the chain and more reliable data transfer, and is less taxing on a CPU."
    http://en.wikipedia.org/wiki/IEEE_1394_interface#Comparison_with_USB
  • »30.09.10 - 10:10
    Profile