Post
Topic
Board Service Discussion
Re: Stay away from ASICSPACE
by
jtoomim
on 06/05/2015, 09:30:52 UTC
Temperature rise is relatively linear even if it doesn't feel like it and the damage caused is not linear.  You noticed around ~95F in the cold aisle @ ~61F outside.  On a 81F day, that would be ~115F(~46C).  Maybe a bit more, but the damage to these miners(and why other miners were not similarly damaged) was primarily because of the airflow around them in relation to the S4's you mentioned, not primarily because of the intake temperatures.
I observed cold aisle temps around 35°C. My estimate of cold aisle temps around 55°C in the weeks before I visited was based on several factors:

1. There were network problems when I visited, resulting in less heat generated. It looked like about 50% of ASICSPACE's miners were off while I was there due to the network problems. If the 50% estimate is correct, then the delta T versus outside might be sqrt(2) to 2 times higher, depending on how much higher inside temperatures assist the stack effect. As the cold aisle temps were already 20°C hotter than outside, this alone might be enough to produce cold aisle temps of 55°C even on a somewhat cool day. However, my 50% estimate could be off. Also, many of the machines were S4s, which may also produce heat while the network is disconnected. For my estimate, I think I was generous and presumed that the network problems reduced temps by 8°C.

2. The nine flexible ducts supplementing the cold aisle with outside air at ambient pressure looked to be hastily added. I presumed they were added within the last week before my visit, and were not present when the damage occurred. I estimate they added about 20% to the airflow into the cold aisles. Roughly speaking, that should have decreased the cold-aisle/outdoor delta T by 20%, which would be about 4°C on the day I visited, or possibly as much as 8°C on other days. There may have been other similar quick fixes that I did not notice. I used 2°C in my quick estimate.

3. When I visited (6pm to 11pm), the outside temp was 16°C or lower. The high the day before was 26°C. That's 10°C. I think there was a day or two in the week or two prior that was even hotter.

8°C + 2°C + 10°C = 20°C
35°C + 20°C = 55°C

Other effects: as the temperatures rose, many machines may have turned off. On the other hand, reliance on the stack effect may have worsened the effects of a high outside temperature.

My math might not be right, of course. It's just an educated guess.

Temperature rise is relatively linear even if it doesn't feel like it and the damage caused is not linear.

The damage we observed was not linear. In terms of deformation, pretty much all of the plastic panels were deformed. Most of the panels were only slightly deformed, maybe 1 or 2 mm per panel. Other panels were very heavily deformed, with about 20 mm of deformation at several different points on each panel. If the maximum temperature seen by each miner was normally distributed, and the amount of deformation as a function of temperature is something like D(T) = e^T, that would fit our observed distribution of deformation pretty closely.

We saw something similar with the amount of electrical damage on each hashboard. Most of the hashboards had either zero bad stages or more than two bad stages. This could also be caused by a domino effect, though.

It's also worth mentioning that the damage to the two hashboards in a single miner were correlated. Most of the miners we received either had two bad hashboards or zero bad hashboards. The number of machines with exactly one bad hashboard was less than I had expected.

Quote
Polyethylene has a glass transition temperature of -80 to -120C.  Polyethylene is malleable at room temperature and cheap to mass produce, just like the S5 plastic, which makes it a reasonable assumption unless Bitmain can confirm.  Regardless, the temperatures that caused the deformation & damage would have largely been due to the "stuck" stagnant air that you described, not a direct result of the hot aisle temperatures. (i.e., primary cause = pressure)  Other minor information I now know that I can't share agrees with that.

I disagree. I do not see how air pressure alone could have caused this deformation. I just went over and tested, and using my finger, it takes about 20 pounds of force to permanently deform a S5 side panel at room temperature by about 3 mm. That would mean you'd need close to 1 atmosphere of pressure difference across the panel to deform it from pressure alone. Typical HVAC system pressure differences are on the order of 100 Pa (0.015 PSI). Axial fans typically have static pressure capabilities around 300 Pa. If we are very pessimistic and say that the hot aisle was 600 Pa higher pressure than the cold aisle due to the two S4 fans (i.e. the S4s had zero net airflow, 100% static pressure), then that would mean about 0.06 newtons per square centimeter, or 0.08 PSI. I think temperature must have been a large factor.

Polylactic acid has a glass transition temperature of 60°C to 65°C. What are the panels actually made of? I don't know. Also, even below or above the glass transition temperature, plastics will become softer and more plastic as temperature increases. The Vicat softening temperature for HDPE is about 70°C, for example. The 0.08 PSI might not be enough to deform it at all at 20°C, but it might be enough to deform it by 1 mm at 70°C, and enough to deform by 10 mm at 80°C. I do not feel inclined to test this right now, but if this goes to court, it would be simple enough to verify.