Ncdu: deleted /tmp, hdd remains full


#1

Trying to resolve what seems to be an inode issue - checked the usual sources (stackoverflow, etc) to no avail. Have tried drilling down on the df output below with “for i in /*; do echo $i; find $i |wc -l; done” and I found /home/ to be 222243! Digging deeper /home/ubuntu/workspace and I don’t see anything remotely close to such size.

Any ideas? Any attempt to install a gem yields the crafty ‘no space on disk…@io_write’ message. Sorry if this question is low-level…

$ df -ih
Filesystem Inodes IUsed IFree IUse% Mounted on
none 291K 210K 81K 73% /
tmpfs 6.4M 18 6.4M 1% /dev
shm 6.4M 1 6.4M 1% /dev/shm
tmpfs 6.4M 11 6.4M 1% /sys/fs/cgroup
/dev/mapper/volg1-lvdata 91M 12M 80M 13% /nix


#2

Try running

sudo du -h -t 50M / --exclude=/nix --exclude=/mnt --exclude=/proc 

or

du-c9 | grep -P '^(\d\d+M|[\d.]+G)'

to see the list of large files.

Usually it’s something in ~ e.g. tmp files from npm or heroku


Workspace is running out of quota
Workspace space
Having a hard time getting free space
Requesting Help with Disk Space Full Error
Workspace is running out of quota
Npm jspm install failed, took all HD space?
I am using 764 out of the 10 GB disk space
Disk space being used but i cannot pin point what is causing it
#3

Thank you. Any suggestions for cleaning this up permanently and preventing future issues? Or is this just required regular maintenance on c9? Truncated output of du -h -t 50M / (for all >1.0G):

3.3G /nix/store
du: cannot read directory ‘/nix/.cache’: Permission denied
3.4G /nix
2.6G /usr
1.8G /home/ubuntu/.local/share/heroku/tmp
1.9G /home/ubuntu/.local/share/heroku
1.9G /home/ubuntu/.local/share
1.9G /home/ubuntu/.local
2.0G /home/ubuntu
2.0G /home
8.8G /


#4

check what you have in /home/ubuntu/.local/share/heroku/tmp
likely most of that can be deleted

is this just required regular maintenance on c9

that depends on what programs you use, if heroku creates too much tmp files, you need to either periodically remove that, or upgrade to a larger workspace.


#5

In the interest of closure…all has been resolved. For anyone else encountering ‘no space on device’ or c9 warnings about disk space usage, here’s the recap:

Disk usage (‘no space on device’ error) - potential solution is to delete heroku/tmp files:

  1. Check for inode usage issue: df -ih
  2. Run du -h -t 50M / (see where the largest files are)
  3. Look for heroku/tmp files that might need to be flushed, then
  4. cd to ~/.local/share/heroku/tmp & use ncdu to delete the files

#6

I’ve follow the commands that you provided precisely (see below).

I don’t have any apps or heroku files but I still have 993MB space occupied…
How to get rid from it?

statyana_pol:~/workspace ls -la total 16 drwxrwxr-x 3 ubuntu ubuntu 4096 Jun 24 17:09 ./ drwxr-xr-x 25 ubuntu ubuntu 4096 Jun 9 22:06 ../ drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 1 07:42 .c9/ -rw-rw-r-- 1 ubuntu ubuntu 948 Jan 18 2016 README.md tatyana_pol:~/workspace df -ih
Filesystem Inodes IUsed IFree IUse% Mounted on
none 291K 75K 216K 26% /
tmpfs 6.4M 17 6.4M 1% /dev
tmpfs 6.4M 15 6.4M 1% /sys/fs/cgroup
/dev/mapper/volg1-lvdata 75M 8.6M 67M 12% /mnt
shm 6.4M 1 6.4M 1% /dev/shm
tmpfs 6.4M 1 6.4M 1% /sys/firmware
tatyana_pol:~/workspace du -h -t 50M / du: cannot read directory ‘/nix/.cache’: Permission denied 55M /nix/store/cz8sfjinal66y350ynyz89vihihcn7k6-gcc-4.9.3/libexec/gcc/x86_64-unknown-linux-gnu/4.9.3 55M /nix/store/cz8sfjinal66y350ynyz89vihihcn7k6-gcc-4.9.3/libexec/gcc/x86_64-unknown-linux-gnu 55M /nix/store/cz8sfjinal66y350ynyz89vihihcn7k6-gcc-4.9.3/libexec/gcc 55M /nix/store/cz8sfjinal66y350ynyz89vihihcn7k6-gcc-4.9.3/libexec 99M /nix/store/cz8sfjinal66y350ynyz89vihihcn7k6-gcc-4.9.3 68M /nix/store/d1prcspbh2qsviipvnaxizcj8l3g7fpw-bootstrap-tools/libexec/gcc/x86_64-unknown-linux-gnu/5.3.0 68M /nix/store/d1prcspbh2qsviipvnaxizcj8l3g7fpw-bootstrap-tools/libexec/gcc/x86_64-unknown-linux-gnu 68M /nix/store/d1prcspbh2qsviipvnaxizcj8l3g7fpw-bootstrap-tools/libexec/gcc 68M /nix/store/d1prcspbh2qsviipvnaxizcj8l3g7fpw-bootstrap-tools/libexec 114M /nix/store/d1prcspbh2qsviipvnaxizcj8l3g7fpw-bootstrap-tools 106M /nix/store/lrd5wjp4151wmflci68i8vd7a9a4lwmb-glibc-locales-2.23/lib/locale 106M /nix/store/lrd5wjp4151wmflci68i8vd7a9a4lwmb-glibc-locales-2.23/lib 106M /nix/store/lrd5wjp4151wmflci68i8vd7a9a4lwmb-glibc-locales-2.23 61M /nix/store/3fyzfkv0k3d1ymfv25d2k7kqbpibhl1r-gccgo49-4.9.3/libexec/gcc/x86_64-unknown-linux-gnu/4.9.3 61M /nix/store/3fyzfkv0k3d1ymfv25d2k7kqbpibhl1r-gccgo49-4.9.3/libexec/gcc/x86_64-unknown-linux-gnu 61M /nix/store/3fyzfkv0k3d1ymfv25d2k7kqbpibhl1r-gccgo49-4.9.3/libexec/gcc 61M /nix/store/3fyzfkv0k3d1ymfv25d2k7kqbpibhl1r-gccgo49-4.9.3/libexec 67M /nix/store/3fyzfkv0k3d1ymfv25d2k7kqbpibhl1r-gccgo49-4.9.3/lib 151M /nix/store/3fyzfkv0k3d1ymfv25d2k7kqbpibhl1r-gccgo49-4.9.3 70M /nix/store/zmisxnl63ica4zaviifixamr3xg27kzp-ruby-2.3.1-p0/share/ri/2.3.0/system 70M /nix/store/zmisxnl63ica4zaviifixamr3xg27kzp-ruby-2.3.1-p0/share/ri/2.3.0 70M /nix/store/zmisxnl63ica4zaviifixamr3xg27kzp-ruby-2.3.1-p0/share/ri 70M /nix/store/zmisxnl63ica4zaviifixamr3xg27kzp-ruby-2.3.1-p0/share 107M /nix/store/zmisxnl63ica4zaviifixamr3xg27kzp-ruby-2.3.1-p0 52M /nix/store/adnbhcy51b4x19fmfci6h0qfy5sarwpr-python-2.7.11/lib/python2.7 54M /nix/store/adnbhcy51b4x19fmfci6h0qfy5sarwpr-python-2.7.11/lib 55M /nix/store/adnbhcy51b4x19fmfci6h0qfy5sarwpr-python-2.7.11 51M /nix/store/7nygbmc03nxm24iz59s6gwwh9j1q5ham-nixpkgs/nixpkgs/pkgs/development/haskell-modules 77M /nix/store/7nygbmc03nxm24iz59s6gwwh9j1q5ham-nixpkgs/nixpkgs/pkgs/development 128M /nix/store/7nygbmc03nxm24iz59s6gwwh9j1q5ham-nixpkgs/nixpkgs/pkgs 134M /nix/store/7nygbmc03nxm24iz59s6gwwh9j1q5ham-nixpkgs/nixpkgs 134M /nix/store/7nygbmc03nxm24iz59s6gwwh9j1q5ham-nixpkgs 100M /nix/store/gphrkb6lzn9bf7db2q1z00ypfdh46lma-node-sources/deps 136M /nix/store/gphrkb6lzn9bf7db2q1z00ypfdh46lma-node-sources 69M /nix/store/l8y2srrkp5fflwph7vq0gllj1k1ai17w-gcc-5.3.0/libexec/gcc/x86_64-unknown-linux-gnu/5.3.0 69M /nix/store/l8y2srrkp5fflwph7vq0gllj1k1ai17w-gcc-5.3.0/libexec/gcc/x86_64-unknown-linux-gnu 69M /nix/store/l8y2srrkp5fflwph7vq0gllj1k1ai17w-gcc-5.3.0/libexec/gcc 69M /nix/store/l8y2srrkp5fflwph7vq0gllj1k1ai17w-gcc-5.3.0/libexec 116M /nix/store/l8y2srrkp5fflwph7vq0gllj1k1ai17w-gcc-5.3.0 68M /nix/store/w1vk7050ib5vi1qlqr80s3hfvx4ni5cm-python3-3.5.1/lib/python3.5 71M /nix/store/w1vk7050ib5vi1qlqr80s3hfvx4ni5cm-python3-3.5.1/lib 72M /nix/store/w1vk7050ib5vi1qlqr80s3hfvx4ni5cm-python3-3.5.1 1.9G /nix/store 2.0G /nix du: cannot read directory ‘/var/lib/php5’: Permission denied du: cannot read directory ‘/var/lib/mysql’: Permission denied du: cannot read directory ‘/var/lib/postgresql/9.3/main’: Permission denied du: cannot read directory ‘/var/lib/sudo’: Permission denied du: cannot read directory ‘/var/lib/mysql-files’: Permission denied du: cannot read directory ‘/var/lib/couchdb’: Permission denied du: cannot read directory ‘/var/lib/polkit-1’: Permission denied 71M /var/lib du: cannot read directory ‘/var/cache/ldconfig’: Permission denied 103M /var/cache/apt/apt-file 103M /var/cache/apt 110M /var/cache du: cannot read directory ‘/var/log/mysql’: Permission denied du: cannot read directory ‘/var/log/apache2’: Permission denied du: cannot read directory ‘/var/log/exim4’: Permission denied du: cannot read directory ‘/var/log/couchdb’: Permission denied du: cannot read directory ‘/var/spool/exim4’: Permission denied du: cannot read directory ‘/var/spool/rsyslog’: Permission denied du: cannot read directory ‘/var/spool/cron/crontabs’: Permission denied 182M /var 62M /home/ubuntu/.local/share/heroku/client/node_modules 96M /home/ubuntu/.local/share/heroku/client 103M /home/ubuntu/.local/share/heroku 103M /home/ubuntu/.local/share 103M /home/ubuntu/.local 184M /home/ubuntu 184M /home du: cannot read directory ‘/etc/ssl/private’: Permission denied du: cannot read directory ‘/etc/couchdb/local.d’: Permission denied du: cannot read directory ‘/etc/polkit-1/localauthority’: Permission denied 182M /usr/bin 63M /usr/local/heroku/node_modules 97M /usr/local/heroku 114M /usr/local/rvm/gems/ruby-2.3.0/cache 558M /usr/local/rvm/gems/ruby-2.3.0/gems 705M /usr/local/rvm/gems/ruby-2.3.0 705M /usr/local/rvm/gems 103M /usr/local/rvm/rubies/ruby-2.3.0/lib 103M /usr/local/rvm/rubies/ruby-2.3.0 103M /usr/local/rvm/rubies 845M /usr/local/rvm 941M /usr/local 328M /usr/lib/x86_64-linux-gnu 93M /usr/lib/valgrind 70M /usr/lib/python2.7 78M /usr/lib/jvm/java-7-openjdk-amd64/jre/lib 78M /usr/lib/jvm/java-7-openjdk-amd64/jre 78M /usr/lib/jvm/java-7-openjdk-amd64 78M /usr/lib/jvm 870M /usr/lib 155M /usr/share/doc 70M /usr/share/emacs/24.3 70M /usr/share/emacs 580M /usr/share 114M /usr/include/boost 151M /usr/include 2.7G /usr du: cannot read directory ‘/lost+found’: Permission denied du: cannot read directory ‘/root’: Permission denied du: cannot read directory ‘/run/exim4’: Permission denied 86M /opt/go/pkg/tool/linux_amd64 86M /opt/go/pkg/tool 148M /opt/go/pkg 253M /opt/go 253M /opt 52M /mnt/shared/benchmark/go 52M /mnt/shared/benchmark 99M /mnt/shared/lib/node_modules 51M /mnt/shared/lib/python2/lib/python2.7/site-packages/codeintel/codeintel2/stdlibs 68M /mnt/shared/lib/python2/lib/python2.7/site-packages/codeintel/codeintel2 87M /mnt/shared/lib/python2/lib/python2.7/site-packages/codeintel 109M /mnt/shared/lib/python2/lib/python2.7/site-packages 110M /mnt/shared/lib/python2/lib/python2.7 110M /mnt/shared/lib/python2/lib 110M /mnt/shared/lib/python2 257M /mnt/shared/lib 52M /mnt/shared/plugins/salesforce.sync/kracht/node_modules 56M /mnt/shared/plugins/salesforce.sync/kracht 74M /mnt/shared/plugins/salesforce.sync 89M /mnt/shared/plugins 411M /mnt/shared 411M /mnt du: cannot read directory ‘/proc/tty/driver’: Permission denied du: cannot read directory ‘/proc/1/task/1/fd’: Permission denied du: cannot read directory ‘/proc/1/task/1/fdinfo’: Permission denied du: cannot read directory ‘/proc/1/task/1/ns’: Permission denied du: cannot read directory ‘/proc/1/fd’: Permission denied du: cannot read directory ‘/proc/1/map_files’: Permission denied du: cannot read directory ‘/proc/1/fdinfo’: Permission denied du: cannot read directory ‘/proc/1/ns’: Permission denied du: cannot read directory ‘/proc/7/task/7/fd’: Permission denied du: cannot read directory ‘/proc/7/task/7/fdinfo’: Permission denied du: cannot read directory ‘/proc/7/task/7/ns’: Permission denied du: cannot read directory ‘/proc/7/fd’: Permission denied du: cannot read directory ‘/proc/7/map_files’: Permission denied du: cannot read directory ‘/proc/7/fdinfo’: Permission denied du: cannot read directory ‘/proc/7/ns’: Permission denied du: cannot read directory ‘/proc/252/task/252/fd’: Permission denied du: cannot read directory ‘/proc/252/task/252/fdinfo’: Permission denied du: cannot read directory ‘/proc/252/task/252/ns’: Permission denied du: cannot read directory ‘/proc/252/fd’: Permission denied du: cannot read directory ‘/proc/252/map_files’: Permission denied du: cannot read directory ‘/proc/252/fdinfo’: Permission denied du: cannot read directory ‘/proc/252/ns’: Permission denied du: cannot access ‘/proc/2134/task/2134/fd/4’: No such file or directory du: cannot access ‘/proc/2134/task/2134/fdinfo/4’: No such file or directory du: cannot access ‘/proc/2134/fd/3’: No such file or directory du: cannot access ‘/proc/2134/fdinfo/3’: No such file or directory 5.7G / tatyana_pol:~/workspace cd ~/.local/share/heroku/tmp
tatyana_pol:~/.local/share/heroku/tmp ncdu bash: ncdu: command not found tatyana_pol:~/.local/share/heroku/tmp ls -la
total 12
drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 12 21:04 ./
drwxr-xr-x 7 ubuntu ubuntu 4096 Jun 21 16:11 …/
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 21 16:11 u/
tatyana_pol:~/.local/share/heroku/tmp $