Re: [gentoo-user] {OT} Allow work from home?

[J. Roeleveld]

On Wednesday, January 20, 2016 01:46:29 AM lee wrote:
> "J. Roeleveld" <joost@××××××××.org> writes:
> > On Tuesday, January 19, 2016 01:46:45 AM lee wrote:
> >> "J. Roeleveld" <joost@××××××××.org> writes:
> >> > On Monday, January 18, 2016 02:02:27 AM lee wrote:
> >> >> "J. Roeleveld" <joost@××××××××.org> writes:

> >> >
> >> > Yes
> >> >
> >> >> That would be a huge waste of resources,
> >> >
> >> > Diskspace and CPU can easily be overcommitted.
> >>
> >> Overcommitting disk space sounds like a very bad idea.  Overcommitting
> >> memory is not possible with xen.
> >
> > Overcommitting diskspace isn't such a bad idea, considering most installs
> > never utilize all the available diskspace.
>
> When they do not use it anyway, there is no reason to give it to them in
> the first place.  And when they do use it, how do the VMs handle the
> problem that they have plenty disk space available, from their point of
> view, while the host which they don't know about doesn't allow them to
> use it?

1 word: Monitoring.
When you overcommit any resource, you need to put monitoring in place.
Then you also need to ensure you have the ability to increase that resource 
when required.

> Besides, overcommitting disk space means to intentionally create a setup
> which involves that the host can run out of disk space easily.  That is
> not something I would want to create for a host which is required to
> function reliably.

The host should not crash when a VM does or when the storage assigned to VMs 
fills up.
If it does, go back to the drawing board and fix your design.

> And how much do you need to worry about the security of the VMs when you
> build in a way for the users to bring the whole machine, or at least
> random VMs, down by using the disk space which has been assigned to
> them?  The users are somewhat likely to do that even unintentionally,
> the more the more you overcommit.

See comment about monitoring.
If all your users tend to fill up all available diskspace, you obviously can 
not overcommit on diskspace.

> > Overcommitting memory is, i think, on the roadmap for Xen. (Disclaimer: At
> > least, I seem to remember reading that somewhere)
>
> That would be a nice feature.

For VDIs, I might consider using it.
But considering most OSs tend to fill up all available memory with caches, I 
expect performance issues.

> >> >> plus having to take care of a lot of VMs,
> >> >
> >> > Automated.
> >>
> >> Like how?
> >
> > How do you manage a large amount of physical machines?
> > Just change physical to VMs and do it the same.
> > With VMs you have more options for automation.
>
> Individually, in lack of a better way.  Per user when it comes to
> setting up their MUAs and the like, in lack of any better way.  It
> doesn't make a difference if it's a VM or not, provided that you have
> remote access to the machine.

This is where management tools come into play. (Same methods apply to physical 
and virtual)

When talking MS Windows, domains with their policies are very useful. Couple 
that with WSUS for the patching and software distribution tools for the 
additional software installs, and you have a very nice setup.

For Linux, I would recommend tools like Ansible or Puppet to control the 
software on the machines.

For any OS, I would prevent my users from installing random software. And what 
is installed, would be mostly pre-configured out-of-the-box.

> When you one VM for many users, you install the MUA only once, and when
> you need to do updates, you do them only once.  When you have many VMs,
> like one for each user, you have to install and update many times, once
> on each VM.

Management tools.

> > Depends on the requirements. It's cheaper then a few hundred seperate
> > windows licenses.
>
> It's still more expensive than one, or than a handful, isn't it?

The same cost applies to running physical boxes instead of VMs.

> > Last time I had to fully reinstall a windows machine it took me a day to
> > do
> > all the updates. Microsoft even has server software that will keep them
> > locally and push them to the clients.
>
> That would be useful to have.  Where could I download that?
>
> Last time I installed a VM, it took a week until the updates where
> finally installed, and you have to check on it every now and then to
> find out if it's even doing anything at all.  The time before, it wasn't
> a VM but a very slow machine, and that also took a week.  You can have
> the fastest machine on the world and Windoze always manages to bring it
> down to a slowness we wouldn't have accepted even 20 years ago.

Google for "WSUS".
It's been around for a very long time now (since 2005).

> >> The hardware has already been replaced, and the problem persists.  Other
> >> machines of identical hardware that don't run xen don't show any issues.
> >
> > I still say the hardware is buggy. With replacing, I meant replace it with
> > different hardware, not a different version of the same buggy stuff.
>
> The hardware is known to be 100% reliable by own experience for over a
> year, for all the machines.  Only when xen is running, the problem shows
> up.
>
> Replacing the machine with another, identical one, allows to rule out
> that the particular machine which was replaced has an issue and was very
> easy to do, so that was a very reasonable second step after trying
> different network cards.  Three different network cards, from three
> different manufactures, lead to the same error message.
>
> Googling the error message shows that quite a few ppl, with entirely
> different hardware, usually not running xen, have had the same message
> with very similar symptoms.
>
> This currently leaves these possibilities:
>
>
> 1.) Xen doesn't work with this hardware.

Xen doesn't touch the network interfaces, it's the host that handles that.

> 2.) The problem might somehow be caused by an SSD.

Network issues caused by SSD?
Sounds like buggy hardware, replace.

> 3.) The error message is actually true and something yet unknown is
>     going on on the network.
>
> 4.) The problem may have been fixed a while ago in the kernel and has
>     not been fixed in the xen kernel.

Xen kernel?
I use hardened-sources or gentoo-sources for the host.
Which Xen kernel are you talking about?

> 5.) The gplpv drivers the VMs use cause the problem.

Contact the developer to get a debug-version.

> 6.) It's an issue with power management since the problem occurs when
>     the machine and the VMs are not used/busy, at night.  Disabling the
>     power management for the network card has not made a difference,
>     though.

Powermanagement of the switches?

> 3.) is currently being worked on.  It needs to be figured out and, if
> there's something weird going on, to be solved in any case.  6.) seems
> unlikely, 1.) and 2.) can be decided when the the hardware is replaced
> with something entirely different, which is the most painful and most
> time-consuming option.  That would leave 4.) and 5.), and 3.) if 3.)
> cannot be resolved.
>
> It's easy to say that "the hardware is buggy".  I'm not convinced that
> it is.  In any case, you can always run into a situation in which xen
> doesn't work as well as you might wish or have experienced so far.

Hardware should do what it's designed to do.
If it can't handle the function it is build for, it's buggy.


> >> It's time consuming when you have to reinstall the VMs to migrate them
> >> to kvm.  And when you don't have the installers of all the software
> >> that's on some of the VMs and can't get them, you either have to run
> >> them without virtio drivers or you can't migrate them.
> >
> > There are Howtos on the internet describing how to migrate VMs from 1
> > technology to another. Shouldn't be too hard.
>
> I looked for them.  Did you find one that tells you how to install
> the virtio drivers on an existing Windoze 7 VM and that actually works?
> It's already very difficult to get rid of gplpv drivers.

The following usually works:
Boot up in safe mode, delete the drivers, reboot, install virtio, reboot.

> > And keeping the installers at hand is, in my opinion, a requirement of
> > sane
> > system management.
> > I have installers for all the versions of software I deal with.
>
> Indeed --- but some predecessor decided not to keep an installer which
> would be required and is now unavailable.  So the only options are to
> leave the VM running under xen or to run it under KVM without virtio
> drivers.  The latter is bad idea because the application the installer
> would be needed for already has severe performance problems built in,
> and making it worse isn't a good idea.

Request installers from the original source?
Or consider it legacy and migrate away soonish.
What is your recovery plan if the server it's on dies a horrible death?

> >> > The biggest reason why I don't use KVM is the lack of full snapshot
> >> > functionality. Snapshotting disks is nice, but you end up with an
> >> > unclean-
> >> > shutdown situation and anything that's not yet committed to disk is
> >> > gone.
> >>
> >> I'm not sure what you mean.  When you take a snapshot while the VM is not
> >> shut down, what difference does it make whether you use xen or kvm?
> >
> > A "snapshot" for KVM is ONLY the disks.
> > With Xen, VMWare and Virtualbox, I can also make a snapshot/copy of what's
> > in memory. It's that which makes the difference.
>
> Is that possible without freezing the VM while you make a snapshot of
> the memory?

No

> If not, how is it so much better than shutting the VM down?

It's faster in most cases.
Exception being, the VM having such a large amount of memory assigned that the 
disk-I/O to store the memory takes longer than a reboot.

Or, in my usual use-case for needing snapshots, is in the middle of a lengthy 
manual process, I want to take a snapshot of the current situation and a 
reboot at that point in time will actually cause issues.
The software being used is usually in memory and a disk-only snapshot (eg. 
system crashed simulation when restoring) will mean I can start over.

> >> >> Then there's the question how well vnc or spice connections work over
> >> >> a
> >> >> VPN that goes over the internet.
> >> >
> >> > VNC works quite well, as long as you use a minimal desktop. (like
> >> > blackbox). Don't expect KDE or Gnome to be usable.
> >> > I haven't tried Spice yet, but I've read that it performs better.
> >>
> >> It's not like you had a choice when you have Windoze VMs.
> >
> > Windows has RDP, which is a lot better than VNC. Especially when dealing
> > with low-bandwidth connections.
>
> Wasn't RPD deprecated earlier in this discussion because it seemed to be
> not sufficiently secure?

Login to the RDP session can be linked to 2FA or smart-cards which need to be 
plugged into the laptop.
Don't ask me how, but I have seen it work.

Couple that with a VPN (Which I consider an absolute must when allowing 
employees to work from elsewhere via the internet).

It can be made more secure than a simple VNC or NX connection.

> >> > That depends on where you are.
> >>
> >> In this country, you have to be really lucky to find a place where you
> >> can get a decent internet connection.
> >
> > Then in your country, working from home might not be the best option.
>
> That probably goes for most countries.

Probably, I tend to only deal with countries in Europe and the US. This does 
make me less clued up of the reality in other regions.

> >> > The company could host the servers in a decent datacentre, which should
> >> > take care of the bandwidth issues.
> >>
> >> And give all their data out of hands?  And how much does that cost?
> >
> > I'm talking about putting your own hardware there, not letting the
> > datacentre company access to the servers.
>
> How could they reside in a datacenter without the ppl there having
> physical access to them?

Locked cages which you provide and control access to?
If you're worried about people sniffing encrypted network traffic, I would say, 
good luck building your own secure WAN.

> >> > For the employees, if they want to work from home, it's up to them to
> >> > ensure they have a reliable connection.
> >>
> >> It is as much problem of the company when they want the employees to
> >> work at home.  And the employees don't have a choice, they can only get
> >> a connection they can get.
> >
> > If the company insists people work from home, they need to ensure the
> > employees have the option for a usable connection. Most companies I deal
> > with leave working from home as an option to the employees.
>
> Sometimes it's not an option, and there isn't anything a company could
> do to improve what internet connection an employee can get, unless
> they'd spend huge amounts of money to put cables or fiber glass into the
> ground, provided that they'd get the permissions for that.

Then the company doesn't have the right to force it onto their employees.

> Sooner or later, it might become very difficult to find anyone who's
> still willing to spend all the time and money it takes to commute, or
> someone who can still afford it at all, and it might become difficult to
> find an employer willing to spend the money it takes to provide the
> employees with offices.

True, but if you end up paying for the privilege to work, why work at all?
If I would end up with a negative balance because my costs are more then my 
income, I would quit on the spot and actually be better off.

> When you consider the enormous amount of resources that are wasted for
> commuting in an economy and that some economies might start to gain an
> advantage over others by letting ppl work from their homes and by thus
> becoming able to make more competitive offers to their customers, you
> might come to think that it won't take very long before almost everyone
> must work at their home.  So this isn't a problem of a company, or some
> companies, it's a problem for all companies and all employees, as it is
> a problem for all economies and all countries.

Countries where the infrastructure already exists will have this as an 
advantage. Currently, these are also the countries with the higher wages.
If low-salary countries want to be able to compete on that level, the leaders 
of those countries should invest in the infrastructure, instead of their own 
fleet of expensive cars, private planes, castles,....

> >> >> It might work in theory.  How would it be feasible in practise?
> >> >
> >> > Plenty of companies do it this way. If you don't want to pay for
> >> > software
> >> > like XenDesktop, you need to do all the work setting it up yourself.
> >>
> >> VNC is somewhat slow over a 1Gbit LAN.  Did they find some way to
> >> overcome this problem?
> >
> > Depends on the settings.
>
> Well, yes, I guess you can send something like 640x480 with some minimum
> content that changes as little as possible with less trouble over an
> internet connection than something one can actually work with.

Try, lower quality, like less colours.
When configured for LAN-settings, a 640x480 VNC session will perform worse then 
a 1280x1024 VNC session configured for 58k.
Difference is, the larger screen looks horrible, but I can still work on it.

> >> This sounds like it is for people with unlimited resources.
> >>
> >> BTW, access a VM through VNC, and you don't even have any way to make
> >> the mouse pointer in the VNC window actually follow the mouse pointer
> >> you're using, which makes it rather annoying to do anything in the VM
> >> you're looking at.  If you found a solution for that, I'd be curious as
> >> to how you solved this problem.
> >
> > There is, it's even documented.
> > I'm assuming you are talking about the VNC-console Xen provides?
> >
> > Configure the mouse to be a tablet in the VM config and the issue
> > disappears.
> Thanks, I can try that.  I haven't seen this documented anywhere yet.

Google for "xen vnc mouse out of sync"

First hit:
http://www.virtuatopia.com/index.php/Xen_mouse_pointer_appears_in_the_wrong_position_in_VNC_console

Second hit:
http://xen.1045712.n5.nabble.com/vnc-mouse-out-of-sync-td2588371.html
(See 1st reply)

--
Joost