# Known Issues

Also check Airtime's / LibreTime's [troubleshooting page](https://libretime.org/manual/troubleshooting/).

## Known log file locations

`/var/log/airtime/pypo-liquidsoap`\
`/var/log/airtime/pypo/notify.log`\
`/var/log/airtime/zendlog.log`

### Status

The [status page in Airtime](http://studio.radioangrezi.de/systemstatus) does not show a status for some services: `airtime_analyzer`, `airtime-playout`, `airtime-liquidsoap`. The services might be running correctly though. You can check via our [own monitoring](http://studio.radioangrezi.de:2812/) or `service SERVICE-NAME status` and `journalctl -u SERVICE-NAME | tail`

## Error 500 / No PHP logs

You would suspect PHP logs in `/var/log/airtime/zendlog.log`. But there is none (or it is not up to date). Check `/var/log/apache2/error.log`: It might show:

```
[Mon Dec 16 16:36:41.959122 2019] [:error] [pid 26398] [client 127.0.0.1:39688] PHP Fatal error:  Uncaught exception 'Zend_Log_Exception' with message '"/var/log/airtime/zendphp.log" cannot be opened with mode "a"' in /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Log/Writer/Stream.php:81\nStack trace:\n#0 /usr/share/airtime/php/airtime_mvc/application/logging/Logging.php(11): Zend_Log_Writer_Stream->__construct('/var/log/airtim...')\n#1 /usr/share/airtime/php/airtime_mvc/application/logging/Logging.php(83): Logging::getLogger()\n#2 /usr/share/airtime/php/airtime_mvc/application/logging/Logging.php(166): Logging::error('Uncaught except...')\n#3 [internal function]: Logging::loggingShutdownCallback()\n#4 {main}\n  thrown in /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Log/Writer/Stream.php on line 81
```

Check the permissions on `/var/log/airtime/zendlog.log`. It should be 0777, probably.

## DNS Problems

2019-05-20

Recently the HfK-network experienced issues with its DNS Servers (172.16.0.1 + 172.16.0.2). Local streaming services (darkice, airtime, icecast) were affecetd. Resolved by adding additional 3rd-party DNS Servers form Digitalcourage (46.182.19.48) + Chaos Computer Club (213.73.91.35) to /etc/netplan/XX.yaml.

## RabbitMQ hostname changes

2019-05-31

form: <https://libretime.org/manual/troubleshooting/>

If the Airtime logs indicate failures to connect to the RabbitMQ server, such as:

```
2013-10-31 08:21:11,255 ERROR - [pypomessagehandler.py : main() : line 
99] - Error connecting to RabbitMQ Server. Trying again in few seconds
2013-10-31 08:21:11,255 ERROR - [pypomessagehandler.py : main() : line 99] - Error connecting to RabbitMQ Server. Trying again in few seconds - See more at: http://forum.sourcefabric.org/discussion/16050/#sthash.W8OJrNFm.dpuf
```

but the RabbitMQ server is running normally, this error might be due to a change in the server's hostname since Airtime installation. Directory names under /var/lib/rabbitmq/mnesia/ indicate that RabbitMQ's database files are organised according to the hostname of the server, for example:

```
rabbit@airtime
```

where the hostname is airtime.example.com. If the hostname has changed, it may be necessary to reconfigure RabbitMQ manually, as follows:

Delete the files in /var/lib/rabbitmq/mnesia/

```
sudo rm -r /var/lib/rabbitmq/mnesia/*
```

Restart RabbitMQ:

```
sudo invoke-rc.d rabbitmq-server restart
```

Enter the following commands to set up authentication and grant permissions. The rabbitmqctl add\_user command requires the RabbitMQ password from the /etc/airtime/airtime.conf file as an argument. The rabbitmqctl set\_permissions command should be entered on one line, with the list of Airtime services repeated three times:

```
rabbitmqctl add_vhost /airtime rabbitmqctl add_user airtime XXXXXXXXXXXXXXXXXXXX
rabbitmqctl set_permissions -p /airtime airtime "airtime-pypo|pypo-fetch|airtime-analyzer|media-monitor"   "airtime-pypo|pypo-fetch|airtime-analyzer|media-monitor"  "airtime-pypo|pypo-fetch|airtime-analyzer|media-monitor"
```

## PHP Errors

2019-06-01

Libretime logs PHP errors into `/var/log/airtime/zendphp.log` You can use Libretimes log controller to add custom log statements to the code. (Native `echo()` and `print_r()` will not help much.) Use `Logging::error()`, `Logging::warn()`, `Logging::info()` or `Logging::debug()` from anywhere in the Libretime codebase.

## Error 500 in airtime-playout (pypo) and `Invalid argument supplied for foreach()` zendphp.log

2019-06-21

### Phenomenon

airtime-playout (`sudo -u www-data python /usr/bin/airtime-playout`) is outputting 500 errors for all API requests:

```
2019-06-01 19:36:06,106 [listenerstat] [ERROR]  Exception: HTTP Error 500: Internal Server Error
2019-06-01 19:38:06,323 [api_client] [ERROR]  HTTP Error 500: Internal Server Error
2019-06-01 19:38:06,373 [api_client] [ERROR]  HTTP Error 500: Internal Server Error
```

zendphp.log is showing errors like:

```
2019-06-01T19:36:06+00:00 ERR (3): studio.radioangrezi.de [ErrorController.php:54 - errorAction()] - An internal application error has occurred.: exception 'ErrorException' with message 'Invalid argument supplied for foreach()' in /usr/share/airtime/php/airtime_mvc/application/models/ListenerStat.php:98
Stack trace:
#0 [internal function]: exception_error_handler(2, 'Invalid argumen...', '/usr/share/airt...', 98, Array)
#1 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Log.php(636): call_user_func('exception_error...', 2, 'Invalid argumen...', '/usr/share/airt...', 98, Array)
#2 /usr/share/airtime/php/airtime_mvc/application/models/ListenerStat.php(98): Zend_Log->errorHandler(2, 'Invalid argumen...', '/usr/share/airt...', 98, Array)
#3 /usr/share/airtime/php/airtime_mvc/application/controllers/ApiController.php(1273): Application_Model_ListenerStat::insertDataPoints(NULL)
#4 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Controller/Action.php(516): ApiController->pushStreamStatsAction()
#5 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Controller/Dispatcher/Standard.php(308): Zend_Controller_Action->dispatch('pushStreamStats...')
#6 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Controller/Front.php(954): Zend_Controller_Dispatcher_Standard->dispatch(Object(Zend_Controller_Request_Http), Object(Zend_Controller_Response_Http))
#7 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Application/Bootstrap/Bootstrap.php(105): Zend_Controller_Front->dispatch()
#8 /usr/share/airtime/php/vendor/zendframework/zendframework1/library/Zend/Application.php(384): Zend_Application_Bootstrap_Bootstrap->run()
#9 /usr/share/airtime/php/airtime_mvc/application/airtime-boot.php(84): Zend_Application->run()
#10 /usr/share/airtime/php/airtime_mvc/public/index.php(68): require_once('/usr/share/airt...')
#11 {main}
`
```

for `ListenerStat.php:98` and `ApiController.php:1870`

### Not-the-solution

* Checked files: `ListenerStat.php` and `ApiController.php` -> POST getParams() always empty. -> no data -> null / true object (which is not iterable).
* Also checked: `airtime-playout.py` / `listenerstat.py` and `api_client.py` -> all not faulty, because dumped (tcpdump) http packets showed correct payload.
* Notices on the side, but not related: `SQLSTATE[55P03]: Lock not available: 7 ERROR: could not obtain lock on row in relation "cc_pref"` ([#465](https://github.com/LibreTime/libretime/issues/465))
* in place updated of Airtime / Libretime will not be easy. ([#623](https://github.com/LibreTime/libretime/issues/623))

### Solution

The hostname and default apache vhost was changed. Since than the error occured. Apache was 301 forwarding requests made to `anregzistudio.hfk-bremen.de` to `studio.radioangrezi.de`. POST data is lost when forwarding!

Put correct hostname in `base_url` of `/etc/airtime.conf`. This would go for a port change as well (e.g. HTTPS)!

```
  3 [general]
  4 api_key = ...
  5 web_server_user = www-data
  6 base_url = studio.radioangrezi.de
  7 base_port = 80
```

### Helpful

* Dump local http requests with payload: `sudo tcpdump -s 0 -A 'tcp dst port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354)' -i lo`
* ["Looks like your POST data get lost when redirecting"](https://stackoverflow.com/a/3654142)
* [Very similar issue: Wrong port, not wrong hostname.](https://github.com/LibreTime/libretime/issues/749)
* [Also simmilar: Wrong hostname in /etc/hosts](https://github.com/LibreTime/libretime/issues/299)

## "Getting information from the server..."

Airtime shows only "Getting information from the server..." on the [stream preferences page](http://studio.radioangrezi.de/preference/stream-setting).&#x20;

```
Traceback (most recent call last):
   File "/usr/bin/pyponotify", line 4, in <module>
     __import__('pkg_resources').run_script('airtime-playout==1.0', 'pyponotify')
   File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 664, in run_script
     self.require(requires)[0].run_script(script_name, ns)
   File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1444, in run_script
     exec(code, namespace, namespace)
   File "/usr/local/lib/python2.7/dist-packages/airtime_playout-1.0-py2.7.egg/EGG-INFO/scripts/pyponotify", line 6
     backupCount=8)
   File "/usr/lib/python2.7/logging/handlers.py", line 117, in __init__
     BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)
   File "/usr/lib/python2.7/logging/handlers.py", line 64, in __init__
     logging.FileHandler.__init__(self, filename, mode, encoding, delay)
   File "/usr/lib/python2.7/logging/__init__.py", line 920, in __init__
     StreamHandler.__init__(self, self._open())
   File "/usr/lib/python2.7/logging/__init__.py", line 950, in _open
     stream = open(self.baseFilename, self.mode)
 IOError: [Errno 2] No such file or directory: '/var/log/airtime/pypo/notify.log'

```

**Resolution**: Log files could not be written. Add log directory and give write permissions.
