OCFS2 crash

When trying to even cd or ls the mounted OCFS2 partition it crashes.  I think this is a combination of VMWare Server's problem and the way I mounted and symlinked to it.

More than anything this shows the problem and lack of forsight with VMWare, but also that OCFS2 is easily crashed if you do strange things.

Output of /var/log/messages for OCFS2

 

Apr 10 15:57:45 localhost kernel: [84331.691258] Modules linked in: vmnet vmci vmmon ocfs2_stack_o2cb ocfs2 ocfs2_stackglue ocfs2_dlmfs ocfs2_dlm ocfs2_nodemanager configfs nls_cp437 cifs af_packet binfmt_misc rfcomm bridge stp bnep sco l2cap bluetooth ppdev ipv6 powernow_k8 cpufreq_ondemand cpufreq_conservative cpufreq_userspace cpufreq_stats cpufreq_powersave freq_table pci_slot video output container wmi sbs sbshc battery iptable_filter ip_tables x_tables ac lp psmouse serio_raw k8temp evdev pcspkr nvidia(P) agpgart shpchp pci_hotplug parport_pc parport snd_hda_intel snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device button i2c_nforce2 isp1760 i2c_core snd soundcore snd_page_alloc ext3 jbd mbcache sd_mod crc_t10dif sr_mod cdrom sg pata_acpi pata_amd ata_generic sata_nv libata ehci_hcd forcedeth ohci_hcd usbcore scsi_mod dock thermal processor fan fbcon tileblit font bitblit softcursor fuse [last unloaded: vmnet]
Apr 10 15:57:45 localhost kernel: [84331.691258]
Apr 10 15:57:45 localhost kernel: [84331.691258] Pid: 11293, comm: vmware-vmx Tainted: P          (2.6.27-11-server #1)
Apr 10 15:57:45 localhost kernel: [84331.691258] EIP: 0060:[<c038d242>] EFLAGS: 00003246 CPU: 1
Apr 10 15:57:45 localhost kernel: [84331.691258] EIP is at _spin_unlock_irqrestore+0x22/0x40
Apr 10 15:57:45 localhost kernel: [84331.691258] EAX: 00003246 EBX: 00003246 ECX: c0b64204 EDX: 00003246
Apr 10 15:57:45 localhost kernel: [84331.691258] ESI: c0b6420c EDI: 00003246 EBP: d6017c94 ESP: d6017c88
Apr 10 15:57:45 localhost kernel: [84331.691258]  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Apr 10 15:57:45 localhost kernel: [84331.691258] CR0: 8005003b CR2: a4706000 CR3: 088f8000 CR4: 000006b0
Apr 10 15:57:45 localhost kernel: [84331.691258] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
Apr 10 15:57:45 localhost kernel: [84331.691258] DR6: ffff0ff0 DR7: 00000400
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<f90e2069>] ocfs2_should_refresh_lock_res+0xf9/0x1c0 [ocfs2]
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c01802c6>] ? handle_edge_irq+0xd6/0x120
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<f90e254d>] ocfs2_inode_lock_update+0xad/0x5c0 [ocfs2]
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c038d1bd>] ? _spin_lock+0xd/0x10
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<f90f50b2>] ? ocfs2_wait_for_recovery+0x32/0xc0 [ocfs2]
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<f90e5b1d>] ocfs2_inode_lock_full+0x37d/0x6f0 [ocfs2]
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<f90e6166>] ocfs2_inode_lock_with_page+0x26/0x80 [ocfs2]
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<f90d35cb>] ocfs2_readpage+0x4b/0x290 [ocfs2]
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c018bdc3>] ? find_get_page+0x23/0xb0
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c018db1b>] do_generic_file_read+0x1cb/0x4d0
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c018dec3>] generic_file_aio_read+0xa3/0x1d0
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c018b960>] ? file_read_actor+0x0/0xe0
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<f90dd62f>] ? ocfs2_inode_unlock+0xff/0x1f0 [ocfs2]
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<f90e941f>] ocfs2_file_aio_read+0xdf/0x460 [ocfs2]
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c01bad89>] do_sync_read+0xd9/0x120
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c023dfb4>] ? aa_file_permission+0x14/0xc0
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c014f040>] ? autoremove_wake_function+0x0/0x50
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c021c494>] ? security_file_permission+0x14/0x20
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c01bae1d>] ? rw_verify_area+0x4d/0xc0
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c01ba8ab>] ? fsnotify_access+0x6b/0x80
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c01bb53d>] vfs_read+0x9d/0x110
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c01bacb0>] ? do_sync_read+0x0/0x120
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c01bb633>] sys_pread64+0x83/0x90
Apr 10 15:57:45 localhost kernel: [84331.691258]  [<c0109f03>] sysenter_do_call+0x12/0x2f
Apr 10 15:57:45 localhost kernel: [84331.691258]  =======================

Restart OCFS2 Service

service ocfs2 restart

Stopping Oracle Cluster File System (OCFS2) Failed
Cannot stat /var/lib/vmware/Virtual\040Machines: No such file or directory
Cannot stat /var/lib/vmware/Virtual\040Machines: No such file or directory
Cannot stat /var/lib/vmware/Virtual\040Machines: No such file or directory
/mnt/sda2:          cmcmcccc
Could not kill process 11592: No such process
Could not kill process 11770: No such process
Could not kill process 12273: No such process
Could not kill process 12360: No such process
/var/lib/vmware/Virtual_Machines:cmcmcccc
Could not kill process 5792: No such process
Could not kill process 11279: No such process
Could not kill process 11592: No such process
Could not kill process 11770: No such process
Could not kill process 11919: No such process
Could not kill process 12273: No such process
Could not kill process 12360: No such process
Retry stopping Oracle Cluster File System (OCFS2) Failed
Cannot stat /var/lib/vmware/Virtual\040Machines: No such file or directory
Cannot stat /var/lib/vmware/Virtual\040Machines: No such file or directory
Cannot stat /var/lib/vmware/Virtual\040Machines: No such file or directory
Retry stopping Oracle Cluster File System (OCFS2) Failed
Cannot stat /var/lib/vmware/Virtual\040Machines: No such file or directory
Cannot stat /var/lib/vmware/Virtual\040Machines: No such file or directory
Cannot stat /var/lib/vmware/Virtual\040Machines: No such file or directory
Starting Oracle Cluster File System (OCFS2) OK

 

Relevant OCFS2 Mount Points & Symlinks

/mnt/sda2/VMWare on /var/lib/vmware/Virtual Machines type none (rw,bind)

I think the space in "Virtual Machines" is half the problem and I was also experimenting on it with new symlinks and even deleting it at some points.  I think this is what did it since that "Virtual Machines" was remounted from the OCFS2 partition.
 


Tags:

ocfs, crashwhen, ls, mounted, partition, crashes, vmware, server, symlinked, forsight, output, var, apr, localhost, kernel, modules, linked, vmnet, vmci, vmmon, _stack_o, cb, _stackglue, _dlmfs, _dlm, _nodemanager, configfs, nls_cp, cifs, af_packet, binfmt_misc, rfcomm, stp, bnep, sco, bluetooth, ppdev, ipv, powernow_k, cpufreq_ondemand, cpufreq_conservative, cpufreq_userspace, cpufreq_stats, cpufreq_powersave, freq_table, pci_slot, container, wmi, sbs, sbshc, iptable_filter, ip_tables, x_tables, ac, lp, psmouse, serio_raw, temp, evdev, pcspkr, nvidia, agpgart, shpchp, pci_hotplug, parport_pc, parport, snd_hda_intel, snd_pcm_oss, snd_mixer_oss, snd_pcm, snd_seq_dummy, snd_seq_oss, snd_seq_midi, snd_rawmidi, snd_seq_midi_event, snd_seq, snd_timer, snd_seq_device, c_nforce, isp, c_core, snd, soundcore, snd_page_alloc, ext, jbd, mbcache, sd_mod, crc_t, dif, sr_mod, cdrom, sg, pata_acpi, pata_amd, ata_generic, sata_nv, libata, ehci_hcd, forcedeth, ohci_hcd, usbcore, scsi_mod, dock, thermal, processor, fbcon, tileblit, font, bitblit, softcursor, fuse, unloaded, pid, comm, vmx, tainted, eip, eflags, cpu, _spin_unlock_irqrestore, eax, ebx, ecx, edx, esi, edi, ebp, esp, ds, es, fs, gs, ss, cr, ffff, ff, _should_refresh_lock_res, xf, handle_edge_irq, xd, _inode_lock_update, xad, bd, _spin_lock, _wait_for_recovery, xc, _inode_lock_full, _inode_lock_with_page, _readpage, bdc, find_get_page, xb, db, do_generic_file_read, dec, generic_file_aio_read, xa, file_read_actor, xe, dd, _inode_unlock, xff, _file_aio_read, xdf, do_sync_read, dfb, aa_file_permission, autoremove_wake_function, security_file_permission, bae, rw_verify_area, ab, fsnotify_access, bb, vfs_read, bacb, sys_pread, sysenter_do_call, restart, oracle, cluster, stat, lib, virtual, directory, mnt, sda, cmcmcccc, virtual_machines, retry, ok, relevant, mount, symlinks, rw, bind, quot, experimenting, deleting, remounted,

Latest Articles

  • How To Upgrade Debian 8,9,10 to Debian 12 Bookworm
  • Linux dhcp dhclient Mint Redhat Ubuntu Debian How To Use Local Domain DNS Server Instead of ISPs
  • Docker dockerd swarm high CPU usage cause solution
  • Docker Minimum Requirements/How Efficient is Docker? How Much Memory Does Dockerd Use?
  • qemu-nbd: Failed to set NBD socket solution qemu-nbd: Disconnect client, due to: Failed to read request: Unexpected end-of-file before all bytes were read
  • apache2 httpd apache server will not start [pid 22449:tid 139972160445760] AH00052: child pid 23248 exit signal Aborted (6) solution Mint Debian Ubuntu Redhat
  • How to use the FTDI USB serial cable to RJ45 adapter to connect to the console on Cisco/Juniper Switch Router Firewall in Linux Ubuntu Debian Redhat
  • How To Setup Python3 in Ubuntu Docker Image for AI Deep Learning
  • How to Configure NVIDIA GPUs with Docker on Ubuntu: A Comprehensive Guide for AI Deep Learning CUDA Solution
  • Linux Ubuntu Mint how to check nameservers when /etc/resolv.conf disabled solution
  • Docker cannot work on other overlayfs filesystems such as ecryptfs won't start overlayfs: filesystem on '/home/docker/overlay2/check-overlayfs-support130645871/upper' not supported as upperdir
  • Linux How To Access Original Contents of Directory Mounted Debian Mint CentOS Redhat Solution
  • ecryptfs how to manually encrypt your existing home directory or other directory
  • How to Reset CIPC Cisco IP Communicator for CME CUCM CallManager
  • Internet Explorer Cannot Download File "Your security settings do not allow for this file to be downloaded." Security Settings Solution
  • Linux How To Upgrade To The Latest Kernel Debian Mint Ubuntu
  • Firefox how to restore and backup saved passwords and history which files/location
  • Linux How To echo as root solution to use tee permission denied solution Ubuntu Debian Mint Redhat CentOS
  • Linux how to keep command line bash process running if you are disconnected or need to logout of SSH remotely
  • Linux swapping too much? How to check the swappiness and stop swapping