On 6/28/20 6:37 AM, echo test wrote:
Hello,
Thank you for all your answers and sorry to be late for answering.
> I prefer ZFS but I find that lots of corps prefer mdadm. I really think that's simply >
because ZFS came from Sun and they lack Solaris backgrounds. Now, in a
low-> RAM environment with simpler disc needs, I would probably go
with mdadm.
> Anything else I would choose ZFS. It's ability to take care of itself is surprisingly
> strong. Less work for me after the set up and installation.
ZFS beeing a filesystem and mdadm an utility software, I think I'll go
for mdadm. I didn't know that Debian was supporting ZFS I always used
Ext4.
It's a stack. You build up from disk, to block-level raid, to volume
manager, to file system, to access protocol.
ZFS includes multiple levels of the stack. And yes there are ZFS
implementations for Debian, along with a dozen or more other file systems.
Sorry if I'm misunderstanding, are you saying that Debian cannot
scale in a bigger enterprise ?
Can you tell me what happened with hardware RAID solutions?
> small" could be anything from 10 to 1000 users. Mentioning some numbers
> could get you more useful recommendations.
> In any case, some interesting hardware not mentioned so far (don't
> forget about the power consumption).
Small here is for me about 2000 users all are restaurants that save
their selling history locally on their own server then 2 or 3 times in
the morning they will rsync their postgres data on my data center.
About the power consumption, any advice about some low power hardware
are also welcome.
Something seems a little flakey about that architecture. Rsyncng
postgress data? There are lots of better ways to merge data into a
database. Particularly in these days of cheap, continuous, broadband
connectivity. And if you can't get business broadband in all your
locations, cellular modems are dirt cheap. (I speak as someone who's
designed more than a few mobile data collection systems - everything
from tactical military systems to transit buses).
> Supermicro 1U servers - run two or more of them
> and it's easy to turn them into a high-available cluster
> ...
> Note: I'm seriously considering migrating from Debian for our
> next refresh - I really don't like systemd - might go all the way to BSD
>or an OpenSolaris distro.
Supermicro seems definitely to propose some great stuff I will take
them in account. Why do you dislike systemd ? I heard many people
saying the same thing and I don't really understand what are their
motivation except initd is less invasive.
It's a spaghetti coded package of crap, that takes over your system and
does things its way. I prefer modularity, and control over my systems.
I don't really know how to answer to your question but let's try. We
are a startup and for the moment we have a production and a
development, in fact the production is just like a test environment
because we do continuous delivery, we push everyday in order to know
more quickly when something has been broken and our semi-automated
tests didn't detect it. Personally, I'm a self learner, and probably
many guys of my team are too. So some advices here are also welcome.
We want to be able to handle 2500+ rsync in the morning (probably
distributing them in time in order to avoid a single big load acting
as a ddos) and for each client of my clients (restaurants) a get and
put profile request.
Note: client's profile are shared across restaurants and clients can
find/filter restaurants on the website which is not yet built but we
are working on it.
Rsync is just wrong for that kind of application. What are you syncing,
anyway. 2500 postgress instances, some raw data files for input to a
single instance of postgress, something else? And if the postgress
instance is feeding live data to apps, you really need to focus first on
your high-availability strategy - single points of failure will kill you.
Come to think of it, you're a poster child for doing everything in the
cloud. As a startup, you've got way too many other things to worry
about than home brewing an IT environment - focus on your core
product/service, whatever that is. (Now, if you're setting up a service
bureau, that's another story - in which case, hire some folks who
actually know how to do this stuff. Here, I'm speaking as someone who
HAS homebrewed a small service bureau, with serious experience in
computing & IT - back before any of this stuff was available off the
shelf. It's a royal PITA. These days, I'm far more likely to set up a
new domain, or app, on a hosting service, than on our cluster - unless &
until I know that it needs to be around for a while. Life's too short.)
Miles Fidelman
--
In theory, there is no difference between theory and practice.
In practice, there is. .... Yogi Berra
Theory is when you know everything but nothing works.
Practice is when everything works but no one knows why.
In our lab, theory and practice are combined:
nothing works and no one knows why. ... unknown
