Amir's Rant #5: the progress bar and 99% problem

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
So here I go again. Hoping to have a relaxing time after Christmas and run into something else that boils my blood :).

I have a new Qnap NAS server that I am installing/initializing. For those of you who don't know what it is, it is a server with storage that sits on your home network and you can access it from both inside your own and increasingly outside.



At first I marveled by the level of polish in the installation software. It is all cloud based with no software to download or install. Simply gave it the code on the box and it started doing its thing with the common first step of updating its firmware and then proceeding to configure the disk array. The former is a clever answer to a manufacturing problem of how to keep the new manufactured devices updated software. You don't have to care about that anymore. You just make sure the box can wake up and fetch its new software. Hence the reason every device these days has a forced firmware update on first power on.

I bought four 6 Terabyte drives for the unit. Yes, 6 Tbyte :D. Life is good. Three will be the primary storage and the other is a "parity" drive that allows one drive to fail and the system still going. Rebuilding with new replacement drive can be lengthy but I wanted maximum storage in this new Qnap T-431 server. So I bought the largest drive I could which was 6 Tbyte. Usage will be for video recording and security cameras.

Anyway, back to my rant. Everything went smoothly and rather quickly. Then came time to configuring the "RAID 5" disk array and it put up a progress bar. The bar jumped to 30%, sat there for a few seconds and them immediately jumped to 99%. Then it sat there, and sat there. It is still stuck at 99% some 10 minutes later. Just checked and it is still at 99%.

Why are these progress bars so unreliable? The answer is that they are mostly fake. What the bar is attempting to do is predict the future. If it says 50%, then it means just as much time is left to finish the rest of the task. The way computers work though, this is an impossible problem. The operating system together with the hardware decide how fast or slow things get done. An app can't determine that time. So what is commonly done is to simply give a percentage of number of steps taken. Let's say you want to copy 100 files. The progress bar simply indicates the *number* of files copied. If all the files are small, the progress bar runs like mad. If it then gets to one huge file at the end, it will get stuck at that one step and keep saying "95%" or whatever the file count says.

There can be more sophistication here. The app could measure the actual throughput of the system and attempt to predict the remaining time better. This is complex but it is better than nothing.

Unfortunately there is another reason you can get the "99%" stuck progress bar. Namely, if the system hangs. For some reason, it is always the last step or so that causes system crashes and hangs. Say you are updating the firmware. The last step is rebooting the box after all the firmware is loaded. Should there be a bug in that firmware update, the box will reboot and crash and not respond. The remote software that is running on your PC or cloud in my case, will sit there stupidly saying "99%" complete.

Either give me progress bar that has some resemblance to reality or nothing. I mean how hard is it to know how long a reboot is supposed to take, double that time and then properly tell me the device is "bricked" (dead and can't be brought back to previous step before dying).

In my case, it is stuck in Formatting. Anyone who has ever touched disk server let alone programmed a NAS, should know that formatting four 6 Tbyte drives is going to take an eternity. Why on earth pretend that is only going to take the last 1% of the time to finish the initial install and configuration? They have tested these drives in the factory so they should know the exact time it takes to format these drives (I bought a drive that was certified to work with the NAS).



"Software can be hard" to write. But this stuff is not that hard. It really isn't.

Just checked and it is still sitting there at 99%. Grrrrrr.... Hoping it is not hung or I will create another rant post here. :D
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
So here I go again. Hoping to have a relaxing time after Christmas and run into something else that boils my blood :).

I have a new Qnap NAS server that I am installing/initializing. For those of you who don't know what it is, it is a server with storage that sits on your home network and you can access it from both inside your own and increasingly outside.



At first I marveled by the level of polish in the installation software. It is all cloud based with no software to download or install. Simply gave it the code on the box and it started doing its thing with the common first step of updating its firmware and then proceeding to configure the disk array. The former is a clever answer to a manufacturing problem of how to keep the new manufactured devices updated software. You don't have to care about that anymore. You just make sure the box can wake up and fetch its new software. Hence the reason every device these days has a forced firmware update on first power on.

I bought four 6 Terabyte drives for the unit. Yes, 6 Tbyte :D. Life is good. Three will be the primary storage and the other is a "parity" drive that allows one drive to fail and the system still going. Rebuilding with new replacement drive can be lengthy but I wanted maximum storage in this new Qnap T-431 server. So I bought the largest drive I could which was 6 Tbyte. Usage will be for video recording and security cameras.

Anyway, back to my rant. Everything went smoothly and rather quickly. Then came time to configuring the "RAID 5" disk array and it put up a progress bar. The bar jumped to 30%, sat there for a few seconds and them immediately jumped to 99%. Then it sat there, and sat there. It is still stuck at 99% some 10 minutes later. Just checked and it is still at 99%.

Why are these progress bars so unreliable? The answer is that they are mostly fake. What the bar is attempting to do is predict the future. If it says 50%, then it means just as much time is left to finish the rest of the task. The way computers work though, this is an impossible problem. The operating system together with the hardware decide how fast or slow things get done. An app can't determine that time. So what is commonly done is to simply give a percentage of number of steps taken. Let's say you want to copy 100 files. The progress bar simply indicates the *number* of files copied. If all the files are small, the progress bar runs like mad. If it then gets to one huge file at the end, it will get stuck at that one step and keep saying "95%" or whatever the file count says.

There can be more sophistication here. The app could measure the actual throughput of the system and attempt to predict the remaining time better. This is complex but it is better than nothing.

Unfortunately there is another reason you can get the "99%" stuck progress bar. Namely, if the system hangs. For some reason, it is always the last step or so that causes system crashes and hangs. Say you are updating the firmware. The last step is rebooting the box after all the firmware is loaded. Should there be a bug in that firmware update, the box will reboot and crash and not respond. The remote software that is running on your PC or cloud in my case, will sit there stupidly saying "99%" complete.

Either give me progress bar that has some resemblance to reality or nothing. I mean how hard is it to know how long a reboot is supposed to take, double that time and then properly tell me the device is "bricked" (dead and can't be brought back to previous step before dying).

In my case, it is stuck in Formatting. Anyone who has ever touched disk server let alone programmed a NAS, should know that formatting four 6 Tbyte drives is going to take an eternity. Why on earth pretend that is only going to take the last 1% of the time to finish the initial install and configuration? They have tested these drives in the factory so they should know the exact time it takes to format these drives (I bought a drive that was certified to work with the NAS).



"Software can be hard" to write. But this stuff is not that hard. It really isn't.

Just checked and it is still sitting there at 99%. Grrrrrr.... Hoping it is not hung or I will create another rant post here. :D

I hope it's not hung up too. I used to be better, but I've completely lost my patience for bad software design. I bought something the other day that, when I tried to install it, it asked by to change settings on my router. I have 2 laptops, an iPad, a PS3 and a PS4 working perfectly on that router with the settings as they are, and I have no intention of messing with them, so I took the damn thing back. I know that probably sounds awfully picky to some, but I'm not a techie, I'm a user, and primary a user of Apple devices running in an Apple environment. When I plug in a device or slip in a disc full of software, I expect to actively change nothing, download nothing. I expect to click yes or proceed 2 or 3 times and get back to using my computer.

If I don't get it, I move on pretty quickly. Yes, I'm spoiled. And yes, I pay a premium for it.

Tim
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
The ?&%$? piece of you know what was hung. Installed the local software. It asked me for password. Would not take the one I had given to the cloud interface. On a whim, tried the default and it took that :(. Tried to format using that tool, it failed there with some random error it would not tell me! :( :(. A few more tries and it kept failing. So I asked it to use 3 out of 4 drives and that worked. :( :( :(

It came up so I gave it a test, only to see it writing at just 2 megabytes a second. Rebooted, hooked up ethernet and now it is running at 30 Mbytes/sec.

And yes Bruce, all along I was thinking of swapping for Synology. This one was on sale so I bought it. But boy, did they go for features instead of simple and intuitive install. No average person can ever hope to get it going as I finally did.
 

Bruce B

WBF Founding Member, Pro Audio Production Member
Apr 25, 2010
7,006
512
1,740
Snohomish, WA
www.pugetsoundstudios.com
And yes Bruce, all along I was thinking of swapping for Synology. This one was on sale so I bought it. But boy, did they go for features instead of simple and intuitive install. No average person can ever hope to get it going as I finally did.

I've had 2 Synology NAS servers over the past 5yr. and they have worked flawlessly... even between MAC and PC.
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
Sounds painful, Amir. Did you ever get the 4th drive working?

Tim
 

Kal Rubinson

Well-Known Member
May 4, 2010
2,360
697
1,700
NYC
www.stereophile.com
The ?&%$? piece of you know what was hung. Installed the local software. It asked me for password. Would not take the one I had given to the cloud interface. On a whim, tried the default and it took that :(. Tried to format using that tool, it failed there with some random error it would not tell me! :( :(. A few more tries and it kept failing. So I asked it to use 3 out of 4 drives and that worked. :( :( :(

It came up so I gave it a test, only to see it writing at just 2 megabytes a second. Rebooted, hooked up ethernet and now it is running at 30 Mbytes/sec.

And yes Bruce, all along I was thinking of swapping for Synology. This one was on sale so I bought it. But boy, did they go for features instead of simple and intuitive install. No average person can ever hope to get it going as I finally did.

I don't have to remind you guys about the low sample size of this survey. For example, I have set up 3 QNAP 5-drive NAS without a burp.
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
I don't have to remind you guys about the low sample size of this survey. For example, I have set up 3 QNAP 5-drive NAS without a burp.

It's not a survey, Kal, just a bad customer experience. And it doesn't take many....

Tim
 

Kal Rubinson

Well-Known Member
May 4, 2010
2,360
697
1,700
NYC
www.stereophile.com
It's not a survey, Kal, just a bad customer experience. And it doesn't take many....
I should have put the word survey in quotes since it really isn't one. I bought the QNAPs because of excellent prepurchase experience with them and, in fact, made the purchase before Synology got around to responding. Since then, they have responded efficiently to my 2 support requests. However, I cannot say if my experience is typical.
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
I should have put the word survey in quotes since it really isn't one. I bought the QNAPs because of excellent prepurchase experience with them and, in fact, made the purchase before Synology got around to responding. Since then, they have responded efficiently to my 2 support requests. However, I cannot say if my experience is typical.
They certainly run neck and neck with Synology when it comes to reputation and feature set. Price was, this was a bargain. It was a holiday special on Amazon for just $260. It cost me $1,100 for the drives so the thing cost almost nothing. I looked at equiv. Synology and they are $450 or so.

Right now I am sticking with it and will contact support to see if they can resolve the formatting problem. I think I am one of the few people who tried to use four 6 Tbyte drives. It says something about a 16 Tbyte limit but I thought it would just truncate the file system to that, not just fail.
 

Kal Rubinson

Well-Known Member
May 4, 2010
2,360
697
1,700
NYC
www.stereophile.com
I think I am one of the few people who tried to use four 6 Tbyte drives. It says something about a 16 Tbyte limit but I thought it would just truncate the file system to that, not just fail.
Oh. Could be an issue, I guess.
 

rblnr

Member Sponsor & WBF Founding Member
May 3, 2010
2,151
291
1,170
NYC/NJ
I hope it's not hung up too. I used to be better, but I've completely lost my patience for bad software design. I bought something the other day that, when I tried to install it, it asked by to change settings on my router. ... When I plug in a device or slip in a disc full of software, I expect to actively change nothing, download nothing. I expect to click yes or proceed 2 or 3 times and get back to using my computer.

If I don't get it, I move on pretty quickly. Yes, I'm spoiled. And yes, I pay a premium for it.

Tim

I've arrived at the same place, close to zero tolerance for crappy software/UX. If I can't solve the problem pretty quickly, it goes back. Stuff like this should just work now.
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
So an update. I deleted all the files on the box and it was still accessing the disk drives like mad. Searching everywhere eventually I found a clue that it was synchronizing to the "cloud." What on earth it was synchronizing I don't know. But it had only gotten to 0.5% progress after so many hours and an empty volume! A reboot did nothing. It would just go nuts again. No amount of googling found any clues as what this feature is, or why it would get stuck in this mode.

On reboot, this was a new one on me where it comes to progress bars. It would pop one up and say "rebooting." The bar would go from zero to max, making you think it is about to reboot but nothing would happen and then it would start over! It must have done that 10 times or more before the box rebooted.

I found the log files and they were cryptic as heck. No additional clues at all as to what was going on.



Well duh, I know the format failed. How about some other information to figure out why it failed? That is the purpose of the log files so that advanced users can figure things out. Yet there is no more information. It even uses the generic name "locahost" instead of the real name of the NAS. Sloppy programming.
 

prerich

Well-Known Member
May 21, 2012
246
10
923
So here I go again. Hoping to have a relaxing time after Christmas and run into something else that boils my blood :).

I have a new Qnap NAS server that I am installing/initializing. For those of you who don't know what it is, it is a server with storage that sits on your home network and you can access it from both inside your own and increasingly outside.



At first I marveled by the level of polish in the installation software. It is all cloud based with no software to download or install. Simply gave it the code on the box and it started doing its thing with the common first step of updating its firmware and then proceeding to configure the disk array. The former is a clever answer to a manufacturing problem of how to keep the new manufactured devices updated software. You don't have to care about that anymore. You just make sure the box can wake up and fetch its new software. Hence the reason every device these days has a forced firmware update on first power on.

I bought four 6 Terabyte drives for the unit. Yes, 6 Tbyte :D. Life is good. Three will be the primary storage and the other is a "parity" drive that allows one drive to fail and the system still going. Rebuilding with new replacement drive can be lengthy but I wanted maximum storage in this new Qnap T-431 server. So I bought the largest drive I could which was 6 Tbyte. Usage will be for video recording and security cameras.

Anyway, back to my rant. Everything went smoothly and rather quickly. Then came time to configuring the "RAID 5" disk array and it put up a progress bar. The bar jumped to 30%, sat there for a few seconds and them immediately jumped to 99%. Then it sat there, and sat there. It is still stuck at 99% some 10 minutes later. Just checked and it is still at 99%.

Why are these progress bars so unreliable? The answer is that they are mostly fake. What the bar is attempting to do is predict the future. If it says 50%, then it means just as much time is left to finish the rest of the task. The way computers work though, this is an impossible problem. The operating system together with the hardware decide how fast or slow things get done. An app can't determine that time. So what is commonly done is to simply give a percentage of number of steps taken. Let's say you want to copy 100 files. The progress bar simply indicates the *number* of files copied. If all the files are small, the progress bar runs like mad. If it then gets to one huge file at the end, it will get stuck at that one step and keep saying "95%" or whatever the file count says.

There can be more sophistication here. The app could measure the actual throughput of the system and attempt to predict the remaining time better. This is complex but it is better than nothing.

Unfortunately there is another reason you can get the "99%" stuck progress bar. Namely, if the system hangs. For some reason, it is always the last step or so that causes system crashes and hangs. Say you are updating the firmware. The last step is rebooting the box after all the firmware is loaded. Should there be a bug in that firmware update, the box will reboot and crash and not respond. The remote software that is running on your PC or cloud in my case, will sit there stupidly saying "99%" complete.

Either give me progress bar that has some resemblance to reality or nothing. I mean how hard is it to know how long a reboot is supposed to take, double that time and then properly tell me the device is "bricked" (dead and can't be brought back to previous step before dying).

In my case, it is stuck in Formatting. Anyone who has ever touched disk server let alone programmed a NAS, should know that formatting four 6 Tbyte drives is going to take an eternity. Why on earth pretend that is only going to take the last 1% of the time to finish the initial install and configuration? They have tested these drives in the factory so they should know the exact time it takes to format these drives (I bought a drive that was certified to work with the NAS).



"Software can be hard" to write. But this stuff is not that hard. It really isn't.

Just checked and it is still sitting there at 99%. Grrrrrr.... Hoping it is not hung or I will create another rant post here. :D
Amir...with your expertise and knowledge, you should have just bought the 6 drives, found an old junker of a PC (that had 6 sata connections on the MB) and built yourself a NAS server using Nas4free!!! ;)
My sever is not as big as yours - but 12 TB is large enough for me for now.
 

prerich

Well-Known Member
May 21, 2012
246
10
923
P.S. my sever is totally silent. I love this thing!!!!!
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
Amir...with your expertise and knowledge, you should have just bought the 6 drives, found an old junker of a PC (that had 6 sata connections on the MB) and built yourself a NAS server using Nas4free!!! ;)
My sever is not as big as yours - but 12 TB is large enough for me for now.
Power consumption is a big deal for me given the 24 hour operation. I have plenty of older PCs around but they will use up as much energy as the cost of this NAS in a year or so!

As I mentioned, this was really an impulse buy. It was discounted heavily so I bought it. I had thought about building a PC based one for a long time and due to above reason, I had hesitated.

A quick update: the problem was indeed exceeding its maximum capacity. Bad software meant that it kept trying to do its thing even though it was never going to work. They suggested I should get the Intel based ones which likely use more power. So for now, I am putting up with it.
 

prerich

Well-Known Member
May 21, 2012
246
10
923
Power consumption is a big deal for me given the 24 hour operation. I have plenty of older PCs around but they will use up as much energy as the cost of this NAS in a year or so!

As I mentioned, this was really an impulse buy. It was discounted heavily so I bought it. I had thought about building a PC based one for a long time and due to above reason, I had hesitated.

A quick update: the problem was indeed exceeding its maximum capacity. Bad software meant that it kept trying to do its thing even though it was never going to work. They suggested I should get the Intel based ones which likely use more power. So for now, I am putting up with it.
10-4 Amir. 24TB is a lot of space ;), but yeah - bad software design indeed. I'm running mine 24/7, I'll know during the spring what type of impact its having on the power bill :) ...that's when the heat is off but the air-conditioner is not quite on yet.
 

amirm

Banned
Apr 2, 2010
15,813
37
0
Seattle, WA
Thanks for the suggestion anyway. How about creating a new thread and explaining a bit about your NAS solution anyway? Which storage format you are using, etc.

Given my attention to power usage, I am not too fond of the RAID modes in the NAS. I like the features from unraid, Windows Home Server, etc. where they perform folder replication and hence unused drives stay unpowered. And should there be a failure, the existing drive data is still readable. I don't need the performance that RAID provides given how fast single drives are these days. Like to see if you have dealt with these issues or not.
 

prerich

Well-Known Member
May 21, 2012
246
10
923
Thanks for the suggestion anyway. How about creating a new thread and explaining a bit about your NAS solution anyway? Which storage format you are using, etc.

Given my attention to power usage, I am not too fond of the RAID modes in the NAS. I like the features from unraid, Windows Home Server, etc. where they perform folder replication and hence unused drives stay unpowered. And should there be a failure, the existing drive data is still readable. I don't need the performance that RAID provides given how fast single drives are these days. Like to see if you have dealt with these issues or not.
I will start a thread this evening Amir :)! Admittedly I'm not as adept as you are (Microsoft genius :D ), however I'm not a slouch either - just not as smart as you ;) !!!!!
 

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing