Kernel Panic

Sami,

Interesting you posted this. I had something similar happen on Sunday night to me. Running cues live (no programming at all) and I lost all my backlights (Studio Spots/Colors), but the front light kept going (they are stationary). I looked down and saw the yellow "!" on the console. Bent down to look at the DP8K and I quickly saw "....attack...." and couldn't catch everything else. I did a quick hard reset and the DP came back just fine and continued on the rest of the night with no issues.

I chalked it up to an ethernet glitch of some type (I'm in a permanent install)..but since it happened to you too, I had to drop my two cents in.

Kernel panics, to my knowledge, have never been reported in the field up to this point so this is the first I have heard of this happening in the field during normal playback. I would like to get copies of both Sami and Jon's show that was running at the time. I might be able to induce the error through overnight testing or just by getting the timing right.

Chris...

I'll send my show when I get in this morning.

Details (typical for a Sunday run):
Console started up at 7am
Show run at 9am-1045am and 1115am-115pm
Sits idle from 130p till 4pm
Show run from 5p-7p
Console shut down around 7:30p

"attack" happened approximately 38 minutes into the 5p show run.

I will send my show tomorrow, when I'm back to work.

Our DP is installed in a theater and has almost always power on (UPS). Sometimes we switch it off (during holidays longer than 2 days). So, it is usually on Mon-Sun 24h per day.

The DP has worked almost perfectly, but three times I have catched it booting itself.
The first time it was doing it before I started the console, second time right after show load (It might had an error, but I didn't catch that, just saw it booting)
Also, once it has had assert failed error -> boot.
The kernel panic which is the latest "glitch" is the only one which has happened during programming/playback.
We are running and rehearsing shows from 10am-10pm Mon->Sat
Two shows are running and we are rehearsing one. All of them are based on the same file (I think Chris has some versions of the running ones already and also the "original")
I will send the one we are rehearsing/programming right now.

I think the common thread here is lengthy up time. I have several tests setup now to try and catch this. I am sure it will be random at best. The problem with long up time testing is that it takes a long time to diagnose. Rest assured we have this under our testing cycle here.

Thanks Chris...I'm in the same boat as Sami.

My DP's never shut down (unless I have to do a hard reset). Console is up and down every night, but the DP's run constantly (along with the entire lighting network).

Should I set it up as standard to shut down everything? It's no big deal...I just have to get the procedures in place and get everyone trained.

If it is becoming a common problem for you Jon then, yes, I would recommend power-cycling the DP8000s every morning before you get started. According to our developers the cause may well have nothing to do with the application and require extensive digging through third party code to resolve. Right now we just need to establish a reliable way to reproduce the issue.

Well, we've had the DP8K for 2 years now I think...and MAYBE it's happened 3 times in that two year period (unsure because I never caught it before this last one). I'm not sure if there IS a reliable way to reproduce it...

To err on the side of safety, I will implement a reboot of the DP with the console into the startup. (As long as they can remember to do it).

That being said, our Sunday schedule wouldn't change (up at 7am and finally down at 9pm). That shouldn't be that big of a deal since we would have rebooted at 7am, correct (in theory of course)?

Jon,

You said:
To err on the side of safety, I will implement a reboot of the DP with the console into the startup. (As long as they can remember to do it).

You could create a startup macro that resets the node(s)... Just keep in mind that the macro will fire after the DP loads the show and is ready. So it might confuse your staff when it is on line for a moment then offline, then back on line.

See...sometimes the easiest implementation eludes me.

I didn't even THINK about doing the macro. I already have a comment macro for startup and I'll just add it to the end!

Thanks Brad!

I think I will do that too.
We have had our DP running since the beginning of August and like I said it had booted/crashed four times. But, nothing serious so far...shows are running fine. I will power cycle it regularly now.

Sami and Jon,

Are you running any networking on the fixture link port of the DP8K?

In my case everything is run via fixture link. All data is sent out as Art-Net and converted to DMX in Art-Net nodes.
In Art-Net network I have switches, Art-Net nodes, one laptop (for monitoring) and two video projectors.

Same here...everything is run on fixture link. Leaving Art-net and going to Pathport nodes running Artnet.

Today, the kernel panic happened again...
I did not power cycle after the previous panic, but did a node reset.
This panic happened when I was just playing back a cue...but I'm not sure was I just pressing play or using goto.

i689.photobucket.com/albums/vv259/Surmuri/Hog/3ab1a573.jpg