Quote from: ASICSPACE on December 10, 2014, 04:16:14 AM

Quote from: Endlessa on December 10, 2014, 12:32:40 AM

Quote from: ASICSPACE on December 08, 2014, 12:48:58 AM

Quote from: Endlessa on December 07, 2014, 10:44:27 PM

No raised flooring? cold air from above?

Do vents all line the exterior walls? Just judging from the garage door, is the cold air entering from ~15-20 ft.?

are you doing hot aisle/cold aisle?

Was this building custom built?

or re-purposed with warehouse racks?

The high ceiling are nice, assuming the cooling intake is all the way up on the ceiling.

We are doing hot/cold aisle. Our engineers assure us the high ceilings and concrete will do a lot of the work dissapating the heat for us.

so I assume then the venting on the side is an attempt to use force airflow for cooling. Each pass over a hot aisle will dilute the cooling to the center most rows. In essence you'll be fighting thermal dynamics and having to over cool (and waste much more energy) attempting to reduce hot spots in the center. Higher hardware failures and reduced performance (some hardware will under-clock itself to reduce temp) in the center of the center aisle (or other areas, depending on how the forced flow venting is positioned). While concrete does have decent thermal conductivity, unless the air you are pumping is warmer than the external temperature, it will actually warm the air coming from the wall adjacent venting (see 4th photo on your initial post). Essentially your venting will be cooling the wall first, then the equipment. Furthermore, the first hot aisle will create thermal turbulence in cooling airflow, thus giving inconsistent results to your cooling attempts. This is why most data centers have moved toward something like what's in this simple image:

now I'm not trying to troll or dig on you, more so trying to give potential improvements, but this would be an issue better dealt with prior to having hardware on the racks which will only create migration complexity in the future. The user experience to watch out for, if this is the going forward scenario, is constant complaints of "I have 4 identical units with 1 (located in a possible hot spot) that consistently under performs."

Also, another consideration, when in a heterogeneous hardware situation (I assume you'll be hosting many different brands) and user controlled overclocking your thermal zones will be in constant flux. I just feel concern that this will create the dreaded whack-a-hot-spot situation many early data centers experienced. Without a view of your actual cooling plan, I can only go by what I see in the photo.

Quote from: Kprawn on December 09, 2014, 08:22:31 AM

Take some pictures of the Cooling planned. Grin

I too would love to see these.

Edit: Great resource if your looking to leverage some free advise http://hightech.lbl.gov/datacenters.html

Wow I'm surprised, you have nailed some of the issues our engineers were scratching their heads over. We have a solution, wait until cooling is fully built and I'll post some pics.

Hi Endlessa,

I ran your thoughts by our top engineer and wanted to share with you the following response:

Your thoughts here are all valid, and indeed, likely correct or improvements. In fact, there's nothing I would even disagree with or argue against, and most of these are things that I myself have considered and either brought up or opted to deliberately not mention. There are a couple of mitigating factors that you may not be aware of:

#1 - The exhaust is all very high up, about 70 feet with a colossal amount of total cubic feet to distribute heat. As you no doubt know, heat rises, such that even in short distances the ambient temperature difference between floor and ceiling can be high. This will strongly work in our favor by causing a large pressure differental between the heat exhausts and the cool intakes(down low) even without fans. The question will be how much, and that is what we are going to find out; it is very difficult to calculate with all the added variables.

#2 - Hot spots. We actually have a *lot* of control over the power and heat density of our equipment. We can respond to hot spots not only by adjusting cooling, but also by adjusting miner's physical locations and/or densities, something datacenters cannot do.

#3 - We are specifically not doing things that datacenters do for a lot of reasons. While it is easy to look at this situation and come up with improvements to efficiency, much more difficult is pricing out the total cost of various options, including labor, later adjustments, failure rates and scenarios, and regulations(Raised floors aren't actually cheap, especially not if built per code), and making the right decision. That isn't to say that your ideas are wrong, or that our process is the best. There are just a lot of things that go into the conclusions. Every large scale Bitcoin mine I am familiar with(many) has been built for far less than even cheap datacenters, even after accounting for inefficiencies and later improvements needed.

Drastically changing our approach so late would cause us to not launch on our promised deadlines for our numerous signed customers. Ultimately though, it doesn't matter. We won't be forced to- we are working with an industry expert with a very large amount of industry cooling experience across lots of industrial and commercial setups, including datacenters. Our agreements are such that they are on the hook- REALLY on the hook, not just some legal we'll-sue-you crap- for any inadequacies of the cooling. With no upper limit on the cost- if they found that a raised floor was necessary, they'd be on the hook for it. Conversely, they made the design decisions regarding this after carefully evaluating the costs. I've reviewed their design carefully(With your same thoughts in mind) and confirmed that any cooling inadequacies that occur can be fixed by scaling and building a few cost-effective components. We are intentionally launching in the dead of winter which will give us ample time to experiment and improve before peak temperatures arrive. We already have improvements planned and specific measurements that will tell us which improvements we need. We also have some climate advantages with our particular setup specifics that work in our favor.

tl;dr: I agree with your thoughts, but it isn't necessary as others are on the hook to ensure temps are appropriate, and plans are in place.