HogIII performance... shaking Movinglights.

PureLitePureLite Registered User
Hello everyone

I just started a tour with 20 Elp60 Led-sticks, five MAC 700 and a few conventionals. That gives me about 1400 DMX channels...since I'm running the LED-sticks in 60channel mode. (Yes I want and I need it that way!).
And there is the problem: first I had everything on one DP, first universe 480xLED, sec. univ. 480xLED, 3rd univ. 240xLED plus ML and conv..... well when I swopped all LED, for example from red to blue, it took the console about a half a second to react, plus on fast programming the console crashed! Nework problems! To slow for rock n'roll!... well ok. I changed from one DP to two DP's, moved one of the LED universes to DP two and I changed the Software from build 1014 to 1053. That helped already a lot, the performance improved.
So far so good, but then: when I ran an effect on the LED's and I did a slow move on the ML they start to shake and do no more nice moves they shake from on position to the next... impossible to accept.
So I changed the universes again, now I run 960chans LED on DP one and the rest on DP two 240LED, plus ML etc. Well the ML are better now, not good but I can live with it, but I'm having the old delay on the LED's again and I have a delay between the two DP's!
Maybe this is all normal, but what can we do about it?

Anyone an Idea?

greetz

Michael

Comments

  • stephlightstephlight Registered User
    edited September 2006
    one more DP?
    All cues are unbloked?
  • teericksonteerickson Registered User, HES Alumni
    edited September 2006
    Michael,

    If you can send me your show file, I'd be glad to take a look at what's going on.

    Please send me the show using the HES upload site at:
    http://upload.highend.com

    Let me know which pages, cuelists, and cues are causing you trouble and I'll see if there's anything that I can do to help.

    Thanks.
  • PureLitePureLite Registered User
    edited September 2006
    Thank you, I will do this as soon as possible.. I left the backup in the truck! :) well....

    greetz

    michael
  • Marty PostmaMarty Postma Registered User
    edited September 2006
    Also, if youhave access to something like a DMX DataLynx where you can monitor the ACTUAL refresh rate on the DMX signal coming out of the DP, this might at least let you know where your biggest areas of "congestion" are.
  • PureLitePureLite Registered User
    edited September 2006
    Marty

    I don't really have access to such a tool at moment, but actually I can "see" quite well where the problem is, don't matter which way I plug the universes.. if I have more then 1000 chans on a DP, it slows down...actually even before 1000...
    I believe the DP's or its software are just to slow to handle 1000 chan. at the same time, its not like ML, with LED's, on ML theres is just view chan. changes on a move or color change... but on LED's theres everything at same time, when I ran an effect for example...
  • teericksonteerickson Registered User, HES Alumni
    edited September 2006
    Michael,

    The key factor for performance isn't how many channels are patched to a DP2000, it's how many crossfades are running concurrently. 4 full universes of LED fixtures running a colour effect will take far more processing power than 4 full universes of media servers that are just playing back videos.

    It's important to try to balance your load between the DP2000s on your network. Wholehog 3 is a distributed realtime processing environment where all devices are communicating with each other to stay in sync. If your DP2000s are unevenly loaded, you will have more issues with heavily loaded DPs being behind others.

    I hope this helps.
  • Marty PostmaMarty Postma Registered User
    edited September 2006
    Michael

    I too have seen this problem before, and I am facing it again on another show I'm working on.

    I've got 600 LED channels loaded across two universes on a single DP. I have a second DP to handle all my MLs, etc.

    When I run really big FX I am seeing some "stutter" in the DMX output on the LED DP. The ML DP is unaffected. Interestingly enough if I run a complicated Effect from the Effects Generator, I see the slowdown occur. When I run a 150+ step chase the output is totally smooth. This is probably related to what Tom is talking about with the number of crossfades occuring at the same time.

    My workaround is to build as many FX as I can using chases vs the FX generator....not ideal I agree, but it seems to help quite a lot for now.
  • teericksonteerickson Registered User, HES Alumni
    edited September 2006
    Marty,

    Can you put one of your LED universes on the other DP?
    As I mentioned above, balancing the load between DPs should help a lot.
  • Marty PostmaMarty Postma Registered User
    edited September 2006
    Tom,

    I tried that (one universe per DP), but it only ended up slowing down the DP with the rest of the rig.:no:

    As I mentioned before, chases seem to fix a lot of the problems if not lessen the stutter.

    I will say that the last time I used this many LED fixtures was about 6 months ago, and the performance improvement in that time has been HUGE!!:) :hogsign:
  • PureLitePureLite Registered User
    edited September 2006
    Tom,

    I finally uploaded my showfile..
    I've head meanwhile a few different Problems on the Hog, one of the major Problem was on the HTP, somehow they get stuck once in a while, so they kind of disappear, they are there, just don't work anymore... sometimes they work still on the flash, but not on the fader... really strange... the only workaround I found during the show (where I can't do a restart) is, to make a copy of the cue in the Q-directory and move them new in the Playback.


    Marty,

    Chases will not work for me.. I bump between waves and strobes and all kind of stuff around... which i could never programm in a chase.. sorry.. I the end of it its all a question of performance from the console, the DP's, the software or whatever.. no more compromises!

    To all,

    Just yesterday and today I'm programming on another show, a quite simple Timecode-thing, I won't even start to tell you about the Problems there, its very bad.. actually its unbelievable! I saw the software on there is still the 1014, I want to change to 1053 today, maybe that helps a little. For the moment is just the the highend-download server down...

    In Switzerland are quite a few HOGIII's running even more the Grand-MA's and soon probably more then HOG2's.
    What I want to say is, I don't want any workarounds anymore.. we need solutions and we need them now! I like the HOGIII, but sometimes it's very hard to stand behind a product with so many problems.. Sorry!

    greetz

    Michael
  • teericksonteerickson Registered User, HES Alumni
    edited September 2006
    Michael,

    I've received your show file. I'll take a look at it and let you know if I find anything interesting. I'll keep an eye out for the HTP problem you described.

    I'm interested in hearing more about the timecode problems you've been seeing. What format was the timecode and how were you getting it to the console? What were the issues that you were having?

    I would highly recommend upgrading your console to v1.3.9 build 1053 software. This release included some significant performance improvements and some important bug fixes. I understand your frustrations and appreciate that you continue to use the console, but I want you to know that our entire development team is working extremely hard to continue making improvements and fixing bugs. I feel we've made very good progess in the past year and I'm very excited about some of the things that we're currently working on.

    Please feel free to continue to give us your feedback here. The opinions of our userbase play an important role in driving the direction of console development.

    Thanks.
  • SourceChildSourceChild Registered User
    edited September 2006
    I remember Sean Hoey's quote "If you've got 1000 blue pars at full and you crossfade to 1000 red pars, then you have 2000 crossfades taking place on a DP2000 and it just dies."
    I usually average between 1500 and 1800 channels on my shows when I integrate theatrical conventionals and LEDs into them. This is not always enough to justify jumping to a second DP but I do notice latencies from time to time.

    Here is a solution I try but you'll have to tell me if it's even practical. Since I can usually tie into ETC net, I'll usually patch in complete distribution on my DP. For example:
    • 300 Desk channels
    • 120 LED channels
    • 12 VL3000s
    • 8 VL 3500s
    • 16 Studio Colors
    • 12 Studio Spots
    • 20 Sea Changers
    So I would configure each output to have the same distribution...
    • 75 DMX
    • 30 LEDs
    • 3 VL3000
    • 2 VL3500
    • 4 Colors
    • 3 Spots
    • 5 Sea Changers
    This way, I figure that since I control fixtures as groups more often than not, a fade will be distributed on all four outputs. Obviously this only works when I'm using ETCnet and can configure any of the 32,768 DMX channels the way I want them.
    In real world applications however, try to split my fixture groups across two universes since it is still somewhat easy to get 5-conductor 5-Pin DMX and all I have to do is insert a little pair swapper in the middle of a truss on my DMX line.

    Does any of this matter much? How much does it matter? Does it even matter that I split my dimmer DMX on multiple DP outputs?
  • teericksonteerickson Registered User, HES Alumni
    edited September 2006
    Todd,

    It isn't really important for you to balance fixtures between outputs on a single DP2000. The important factor is how many concurrent crossfades are running on a single DP2000 and balancing the number of crossfades between multiple DP2000s. Most people use a combination of dimmers, automated luminaires, LEDs, and media servers. In these situations, they generally aren't crossfading every parameter of every fixture at the same time and they don't have any problems with workload on their DP2000s.

    The current trend of large rigs with many universes of LED fixtures forces us as programmers and system designers to be more careful with how we patch our rigs. There are two main reasons for this:

    1) LED fixtures have a very small channel count and can be densely packed into universes and all of their channels are usually being crossfaded. Is it very common to run colour effects on large rigs of LED fixtures. It's possible to get into a situation where you have as many as 2048 crossfades running at once and this takes a lot of processing power.

    2) The Wholehog 3 network architecture distributes crossfades. Each DP2000 processes the crossfades and conversion from real-world values to DMX values. This is a *large* advantage of our system, because it means that when you add 4 universes of DMX by adding a DP2000, you're adding the processing power to handle those universes of DMX. Processing DMX and crossfades is a processor-intensive and time-critical job. By distributing these tasks, we offer better performance and response at the console and a far more scalable system. The important thing to think about as a user is that you are no longer working with a system where all DMX is generated on a single PC board with a single clock. There are now independent devices that each generate DMX for a portion of a lighting rig. These nodes communicate with each other in a number of ways to maintain synchronization, but if one DP2000 has a very heavy processing load and another has a very light load, you probably aren't getting the best performance available from your system and you may see the effects of this.

    In summary, balancing the load between outputs on a single DP2000 isn't going to make much of a difference, because all of the crossfades are still being handled by the same processor. Balancing the loads (of simultaneously running crossfades) between DP2000s may make a big difference, but it will depend on your individual rig.

    I hope this helps.
  • SourceChildSourceChild Registered User
    edited September 2006
    Having enough knowledge of general hardware architecture, I can understand the operation of the DP2000, I just wasn't sure if each universe has a discrete processor or if the load was distributed.
    Based on what you're saying, I would suppose the processing system is configured where the DP actually processes a DMX string with two additional bits (making it 1024) and then the output processor just truncates the extra bits before spitting out the RS422 we call DMX. I suppose it doesn't really matter now that I know.

    I've got the whole concept of serial data and I am familiar with dynamic (RYSC) protocol standards.

    Of course now the big question... As a a former linux and UNIX hacker, I am completely familiar with the concept of clustering. What will the future hold for us Hoggies as far as clustering multiple DMX processors together to enhance reliability and performance? So if I were to say link 4 DMX processors together to run 12 universes with load balancing on a 4th processor (hot spare).
  • PureLitePureLite Registered User
    edited September 2006
    Dear Tom

    finally I found time to reply and describe my latest expierences, its always hard to remember everything, when your back home.

    I'running at that point build 1053.

    Timecode: I'm using the USB-TLC interface, which works fine. I'm receiving EBU25 from a Lasercomputer, there is no problem so far. What happend, when I'm running the show, if there is a fadetime, which overlaps the next cue... you understand? For example: CUE27 fadetime20s ,some delay on the gobo or whatever.. TLC 00:01:01:00 ---> CUE28 anytime TLC 00:01:15:00 and maybe there is also couple off effects running..... (lets call it a real world situation), what happens then is that CUE27 freezes or it just doesn't finish or it just doesn't do all the things in CUE28, which are supposed to happen. It's hard to explain. Again: if I have overlapping fadetimes with timecodeinformation, so that a next cue starts before the old is finished, which is possible if they don't do the same parameters or just some of it... THEN i loose some information for the next cue.. You understand?
    Whatsoever, this situation, happens only when I run TC, external or internal does not matter.. when I run the cuelist manually, then there is no problem. I changed the the the whole fade- and delaytiming to a simpler way.. so that are not to many parallel fadings are running, then i twas ok...another compromis..

    Somehow sometimes the playback freezes, no more Go, choose or release, but HTP's are still running. I can not really tell why this happens, mostly during programming, while updating cues. When I restart the playback, it solves usually the problem for a little while, and then it croackes all the way...--> Restart!
    Actually I had to restart the console about every hour, customers asking me if this normal? What answer would you suggest?

    Ah yes and when pressing: OPEN--> PARKED leads me to a editor with the parked channels.. very good Idea! But then I wanted to knockout right from there the parked channels --> Fatal error and goodbye --> restart! Maybe I shouldn't do this...

    greetz

    michael

    PS: I would really like a "Marc-cue" button and anyone who doesn't like it: DON'T USE IT!
  • teericksonteerickson Registered User, HES Alumni
    edited September 2006
    Michael,

    Is this the pluesch06worb.hog3.tar.gz show file that you sent me?

    I loaded this up to try to reproduce your timecode problem here, but there doesn't seem to be any timecode iniformation in the show file.

    Thanks.
  • PureLitePureLite Registered User
    edited September 2006
    The Plüsch file is the one with the LED's, the time u asked me for the file I didn't even started the timecode show yet ... different thing...the timecode problem has nothing to do with that.

    Thats the one with the LED crossfading and HTP-trouble..

    thanks

    michael
  • teericksonteerickson Registered User, HES Alumni
    edited September 2006
    Okay. Thanks.

    I tried to reproduce the timecode problem with a new show, but I wasn't seeing the problems that you were. I'll spend some time digging into this issue and see if I can figure out what the key is.

    Thanks.
  • PureLitePureLite Registered User
    edited October 2006
    Dear Tom

    thanxs.. for spending some time wirh the TC-problem, I'can't send you a show file of this, because I only have a backup on ZIP of this. This show is already over anyway...

    My bigger problem is still the HTP-cues, which are freezing once in a while and without restarting the console they don't come back... as I already described in post#11... It's very scary if you don't know during a show, when you are loosing cues the next time... it just happened on saturday again to me.

    greetz

    michael
  • teericksonteerickson Registered User, HES Alumni
    edited October 2006
    Michael,

    We're looking for the problem you describe, but I don't think I've seen exactly what you had run into. We've been doing lots of work with fader behavivour as we get ready to release the expansion wing and we've resolved a few issues that we've found.

    We'll definitely still be looking for this, so please let us know when it happens and what the surrounding circumstances are.

    Thanks.
  • PureLitePureLite Registered User
    edited October 2006
    I'll have another show tonight... I will try to reproduce my problem in the afternoon.. we'll see...

    michael
  • PureLitePureLite Registered User
    edited October 2006
    tom

    I found the HTP-thing, if you take the show-file I send you, there is a song called "heiweh". So cue 4 is "called swop led chase" now take cue 6,7,8 which are HTPs and hammer like a maniac between the HTPs and the Swop-cue... after a little while you will expierence that the HTPs only on the flashs and not on faders anymore work and when u hit the swop again those HTP will jump to 100% and when you hit the flashs of the HTP again... they will jump to zero.
    When I restart the playbacks, it will solve the problem for a little while...but soon it will return...

    greetz

    michael
Sign In or Register to comment.