Category Archives: Miscellaneous

ISPConfig 3 Showing Default Site

Miscellaneous, , , , ,

After a fresh install of ISPConfig 3 I added some site. Now the strange problem was HTTP version was always opening the default Apache page. But the HTTPS version was opening the proper site.

Solution:

Disable the default website

a2dissite 000-default.conf
The reason for HTTP version opening default site and HTTPS opening proper site:

When Apache is installed it installs a default website (normally /etc/apache2/sites-enabled/000-default.conf). This default website doesn’t have any HTTPS version, only the HTTP version is installed.

Now when the HTTP version of my site was being requested it was opening the HTTP version of the default site as the HTTP version was existing. But as there was no HTTPS version of the default site so when the HTTPS version  of my site was being called no default site was matching and hence it opened the proper site.

Pure-FTPd Error on Amazon EC2

Miscellaneous, , , , , , , , ,

Pure-FTPd error “500 I won’t open a connection to <IP ADDRESS>” OR “Server sent passive reply with unroutable address. Using server address instead.”

This happens when the server is beyond a NAT like the Amazon EC2. The most posted solution on the internet is to fallback to “Passive” mode in the FTP client. But in my case that didn’t help and still I got the same error. After more digging found the solution.

Need to create two files ForcePassiveIP and PassivePortRange and put the port range and the Public IP of the server.

echo "40110 40210" > /etc/pure-ftpd/conf/PassivePortRange 

echo "1.2.3.4" > /etc/pure-ftpd/conf/ForcePassiveIP

1.2.3.4 is the external IP address of the EC2 instance.

Didn’t make any changes to /etc/pure-ftpd/pure-ftpd.conf. Specially didn’t restrict or set the “IPv4 Only “. With IPv4 only I face problems with some internet connections which uses IPv6.

Settings for FileZilla

Encryption: require explicit FTP over TLS
Transfer mode: Passive (PASV)

Restart Pure-FTPd. The command may vary based on which package has been used.

service pure-ftpd-mysql stop
service pure-ftpd-mysql start

Hope this helps someone.

Adding a second disk and quota in Amazon EC2

Miscellaneous, , , , , , ,

This article is based on Ubuntu 20.04.

AWS has good documentation on how to create and add the disk to the instance so not mentioning those steps here.

Once the disk is added to the system verify that it is added and what is the name it is being shown as.

Steps for adding the disk to the system – that is making it ready for mounting

  1. lsblk  --- to check the disk has been added and the name
  2. mkfs -t ext4 /dev/<disk name>    --- Please Note - xfs disks gives error when quota options are added in the fstab files. e.g - mkfs -t ext4 /dev/nvme1n1
  3. mount /dev/<disk name> <mount point>  ---  e.g - mount /dev/nvme1n1 /data
  4. blkid --- check the UUID of the disk
  5. Add entry to fstab

UUID=axxf131c-xxxx-xxxx-8xxx-ec978dxxxxxx /data ext4 defaults,nofail 0 2

Replace /data with your mount point name

nofail will ensure that the system boots even if the disk mounting fails (like when disk removed)
  6. umount <mount point>  ---  umount /data
  7. mount -a  --- Please note - wrong entry in the fstab can make the system unbootable. So please ensure that there are no errors. And don't reboot without resolving errors. 

 

Now the steps for turning on Quota

  1. mount -o remount <disk partition>  e.g - mount -o remount /data
  2. quotacheck -avugm
  3. quotaon -avug
  4. Edit the fstab file and add the necessary
    • nano /etc/fstab
    • add the following to the end of the existing parameters ,usrjquota=quota.user,grpjquota=quota.group,jqfmt=vfsv0
    • Save the file
    • Please note - wrong entry in the fstab file can make the system unusable. Please check the file first before rebooting.  Use mount -a to check if the entries in fstab file is good.

Example:

Disk Quota on Amazon EC2

Miscellaneous, , , ,

The Linux AMIs available for AWS EC2 may not have the packages required for activating Disk Quota.

This article is based on Ubuntu 20.04.

There are many articles and suggestions on the internet. Following one of the suggestions caused my system to become read-only. Lastly found a solution which looks the most legit, simple and works perfect.

It needs installation of the package “linux-modules-extra-aws”

  1. apt-get install linux-modules-extra-aws
  2. apt-get -y install quota quotatool
  3. mount -o remount <disk partition>  e.g - mount -o remount /
  4. quotacheck -avugm
  5. quotaon -avug
  6. Edit the fstab file and add the necessary
    • nano /etc/fstab
    • add the following to the end of the existing parameters ,usrjquota=quota.user,grpjquota=quota.group,jqfmt=vfsv0
    • Save the file
    • Please note - wrong entry in the fstab file can make the system unusable. Please check the file first before rebooting.  Use mount -a to check if the entries in fstab file is good.

XFS in AWS EC2 for MongoDB. The easy way

Miscellaneous, , , ,

Changing the filesystem on AWS EC2 is a difficult task. The easy way to use XFS filesystem on AWS EC2 for hosting MongoDB is to add an extra block storage and format it as XFS

  • First create a Block Storage and attach.
  • Provisioned IOPS SSD is a suitable one for high volume data flow.
  • Once the block storage is attached, then use the command prompt or terminal to format the disk as XFS and mount
    1. apt-get update
    2. apt-get upgrade
    3. lsblk —- to view the attached block device and confirm
    4. file -s /dev/nvme1n1 — to check if there is a filesystem already there or not. if there is no filesystem then it will return “device” . Else it will show something like this “/dev/nvme1n1: SGI XFS filesystem data (blksz 4096, inosz 512, v2 dirs)”.
    5. mkfs -t xfs /dev/nvme1n1 —– command to format the disk with XFS filesystem
    6. apt-get install xfsprogs –— command to install xfs tools
    7. mkdir /mongodata
    8. mount /dev/nvme1n1 /data
    9. mount /dev/nvme1n1 /mongodatabases/
    10. cp /etc/fstab /etc/fstab.orig
    11. blkid —– to check the UUID of the block device added
    12. nano /etc/fstab —– and add the device like this UUID=7xxf03xx-6xxx-4xxx-9xxx-exxxxff2xxxxf /mongodatabases xfs defaults,nofail 0 2
    13. umount /mongodatabases/ —– unmount and mount -a to check fstab entry
    14. mount -a
    15. reboot — I did a double check by rebooting and checked if the disk is still attched and the mounted
    16. df -h
    17. lsblk
    18. mount

Website Push Notification with Google Firebase

Miscellaneous, PHP, , , , , , , ,

Mainly three things are needed

  • A file named firebase-messaging-sw.js placed in the root of the website. This file contains the codes for receiving the push when the browser is minimized or that particular website is closed.
  • The scripts for receiving the push
  • The server side code to send the PUSH

 

firebase-messaging-sw.js

//Base library
importScripts("https://www.gstatic.com/firebasejs/7.21.1/firebase-app.js");

//needed for PUSH Message
importScripts("https://www.gstatic.com/firebasejs/7.21.1/firebase-messaging.js");


// Your web app's Firebase configuration
//All these details is available from Firebase Console. The readymade configuration code is available also - please see the attached screenshot
var firebaseConfig = {
apiKey: "xxxxxxxxxxxxxxxxxxx_x-xxxxxxxxxxxx",
    authDomain: "app-name-123456.firebaseapp.com",
    databaseURL: "https://app-name-123456.firebaseio.com",
    projectId: "app-name-123456",
    storageBucket: "app-name-123456.appspot.com",
    messagingSenderId: "123456789012",
    appId: "1:123456789012:web:xxxxxxxxxxxxxxxxxx",
    measurementId: "G-xxxxxxxxxxxx"
};
// Initialize Firebase
firebase.initializeApp(firebaseConfig);

const messaging = firebase.messaging();

// If you would like to customize notifications that are received in the
// background (Web app is closed or not in browser focus) then you should
// implement this optional method.
// [START background_handler]
messaging.onBackgroundMessage(function(payload) {
  console.log('[firebase-messaging-sw.js] Received background message ', payload);
  // Customize notification here
  const notificationTitle = payload.notification.title;
  const notificationOptions = {
	body: payload.notification.body,
	icon: 'https://mywebsite.com/icon.png',
	image: 'https://mywebsite.com/logo.png'
  };

  return self.registration.showNotification(notificationTitle,
	notificationOptions);
});
// [END background_handler]

Code for Receiving the Push and processing and displaying the PUSH when the page is open

At present some browsers need user interaction (request raised from an user event handler) to show the access permission popup. The best way to do is to check for permission status (granted or not) and show a Custom popup if not granted. In the custom popup put two buttons – Accept/Yes and Decline/No. On pressing Accept/Yes call the requestPermission function.

When the page is fully loaded (document ready) we are calling the updateUIForPushPermissionRequired function. Inside it we are checking if permission already granted or not. If not granted then we are showing the custom popup


<!DOCTYPE html> 
<html> 
  <head> 
    <meta charset="UTF-8"> 
  </head> 
  <body>
    <div id="notificationPopUp" class="notificationPopup" style="display:none"> 
      <div class="notif-body"> 
        <button id="closeNotification" class="uk-modal-close-default uk-icon uk-close" type="button" uk-close="" onclick="dismissnotificationPopUp()"> 
          <svg width="14" height="14" viewBox="0 0 14 14" xmlns="http://www.w3.org/2000/svg" data-svg="close-icon"> 
            <line fill="none" stroke="#000" stroke-width="1.1" x1="1" y1="1" x2="13" y2="13">
            </line> 
            <line fill="none" stroke="#000" stroke-width="1.1" x1="13" y1="1" x2="1" y2="13">
            </line> 
          </svg> 
        </button> 
        <div class="notif-logo">
          <img src="https://api.wisck.com/logo-wisck.png" alt="" width="100%" height="150">
        </div> 
        <div class="notif-content"> 
          <p class="notif-title">Allow Notifications?
          </p> 
          <p class="text_pop_up">Get notifications of new contents and updates
          </p> 
          <p class="text_pop_up">Notification option can be managed from browser settings.
          </p> 
          <div class="notif-btns"> 
            <button id="blockNotification" class="button-decline" onclick="dismissnotificationPopUp()">No
            </button> 
            <button id="allowNotification" class="button-accept" onclick="requestPermission()">Yes
            </button> 
          </div> 
        </div> 
      </div> 
    </div> 

   <div id="snackbar"><p id="snackbarTitle">Title</p><p id="snackbarBody">Body</p></div>

    <!-- The core Firebase JS SDK is always required and must be listed first --> 
    <script src="https://www.gstatic.com/firebasejs/8.6.8/firebase-app.js">
    </script>
    <script src="https://www.gstatic.com/firebasejs/8.6.8/firebase-messaging.js">
    </script> 
    <!-- TODO: Add SDKs for Firebase products that you want to use https://firebase.google.com/docs/web/setup#available-libraries --> 

<script>  // Your web app's Firebase configuration var
    firebaseConfig = {
        apiKey: "xxxxxxxxxxxxxxxxxxx_x-xxxxxxxxxxxx",
        authDomain: "app-name-123456.firebaseapp.com",
        databaseURL: "https://app-name-123456.firebaseio.com",
        projectId: "app-name-123456",
        storageBucket: "app-name-123456.appspot.com",
        messagingSenderId: "123456789012",
        appId: "1:123456789012:web:xxxxxxxxxxxxxxxxxx",
        measurementId: "G-xxxxxxxxxxxx"
    }; // Initialize Firebase 
firebase.initializeApp(firebaseConfig);
firebase.analytics();
const messaging = firebase.messaging();
messaging.onMessage((payload) => {
    console.log('Message received. ', payload);
    var x = document.getElementById("snackbar");
    // Add the "show" class to DIV 
    x.className = "show";
    $("#snackbarTitle").html(payload.notification.title);
    $("#snackbarBody").html(payload.notification.body);
    // After 3 seconds, remove the show class from DIV 
    setTimeout(function() {
        x.className = x.className.replace("show", "");
    }, 3000);
    // Update the UI to include the received message. 
});

function resetUI() {
    // Get registration token. Initially this makes a network call, once retrieved // subsequent calls to getToken will return from cache. 
    messaging.getToken({
        vapidKey: 'xxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-xxxxx-xxxxxx-xxx'
    }).then((currentToken) => {
        if (currentToken) {
            sendTokenToServer(currentToken);
        } else {
            // Show permission request. 
            console.log('No registration token available. Request permission to generate one.');
            // Show permission UI. 
            updateUIForPushPermissionRequired();
            setTokenSentToServer(false);
        }
    }).catch((err) => {
        console.log('An error occurred while retrieving token. ', err);
        //showToken('Error retrieving registration token. ', err); 
        setTokenSentToServer(false);
    });
}
// Send the registration token your application server, so that it can: // - send messages back to this app // - subscribe/unsubscribe the token from topics function 
sendTokenToServer(currentToken) {
    if (!isTokenSentToServer()) {
        console.log('Sending token to server...');
        // TODO(developer): Send the current token to your server. 
        $.ajax({
            url: base_url + 'update-token.php',
            type: 'post',
            data: {
                userId: userId,
                deviceToken: currentToken,
                deviceType: "web"
            },
            headers: {
                'Content-Type': 'application/x-www-form-urlencoded'
            },
            dataType: 'json',
            headers: {
                'Content-Type': 'application/x-www-form-urlencoded',
                'Access-Control-Allow-Origin': '*',
                "Access-Control-Allow-Headers": "Content-Type, Access-Control-Allow-Headers, Authorization, X-Requested-With"
            },
            success: function(response) {
                console.log('Token sent to server...');
            }
        });
        setTokenSentToServer(true);
    } else {
        console.log('Token already sent to server so won\'t send it again ' + 'unless it changes');
    }
}

function isTokenSentToServer() {
    return window.localStorage.getItem('sentToServer') === '1';
}

function setTokenSentToServer(sent) {
    window.localStorage.setItem('sentToServer', sent ? '1' : '0');
}

function requestPermission() {
    $('#notificationPopUp').hide();
    //hide the own permission popup - the browser will take over now 
    console.log('Requesting permission...');
    Notification.requestPermission().then((permission) => {
        if (permission === 'granted') {
            console.log('Notification permission granted.');
            // TODO(developer): Retrieve a registration token for use with FCM. 
            // In many cases once an app has been granted notification permission, 
            // it should update its UI reflecting this. 
            resetUI();
        } else {
            console.log('Unable to get permission to notify.');
        }
    });
}

function deleteToken() {
    // Delete registration token. 
    messaging.getToken().then((currentToken) => {
        messaging.deleteToken(currentToken).then(() => {
            console.log('Token deleted.');
            setTokenSentToServer(false);
            // Once token is deleted update UI. 
            resetUI();
        }).catch((err) => {
            console.log('Unable to delete token. ', err);
        });
    }).catch((err) => {
        console.log('Error retrieving registration token. ', err);
        showToken('Error retrieving registration token. ', err);
    });
}

function updateUIForPushPermissionRequired() {
    if (Notification.permission != "granted") //if permission not granted yet then show custom popup.
    {
        $('#notificationPopUp').slideDown("fast");
        //show the custom popup to ask permission. 
    } else requestPermission();
}

function dismissnotificationPopUp() {
    $('#notificationPopUp').hide();
}
$(document).ready(function() {
        updateUIForPushPermissionRequired();
    }) 
</script> 
   
  </body> 
</html>

CSS for for the notification popup and the notifications

.notificationPopup {
position: fixed;
top: 0;
left: 50%;
transform: translateX(-50%);
background: #fff;
z-index: 9999;
box-shadow: 1px 2px 10px rgba(0,0,0,.5);
}

.notificationPopup .notif-body {
padding: 15px;
display: flex;
}

.notificationPopup .uk-modal-close-default {
position: absolute;
right: 10px;
top: 10px;
background: 0 0;
border: 0;
}

.notificationPopup .notif-logo {
max-width: 70px;
min-width: 70px;
padding-right: 10px;
}


.notificationPopup .notif-logo img{
margin-top: 50%;
}


.notificationPopup .notif-content .notif-title {
font-size: 18px;
font-weight: 600;
margin: 10px 0;
}

.notificationPopup .notif-content .text_pop_up {
font-size: 16px;
font-weight: 400;
margin: 0;
}

.notificationPopup .notif-btns {
text-align: right;
padding-top: 5px;
}

.notificationPopup .notif-btns .button-decline {
border: 1px solid #ddd;
color: #666;
}

.notificationPopup .notif-btns .button-accept {
border: 1px solid #ec2436;
background-color: #ec2436;
color: #fff;
font-weight: 600;
}

.notificationPopup .notif-btns button {
background: 0 0;
border: 0;
padding: 5px 15px;
cursor: pointer;
margin-left: 10px;
}

.notificationPopup .uk-modal-close-default {
position: absolute;
right: 10px;
top: 10px;
background: 0 0;
border: 0;
}

.uk-close {
color: #999;
transition: .1s ease-in-out;
transition-property: all;
transition-property: color,opacity;
}


/* The snackbar - position it at the bottom and in the middle of the screen */
#snackbar {
visibility: hidden; /* Hidden by default. Visible on click */
min-width: 250px; /* Set a default minimum width */
margin-left: -125px; /* Divide value of min-width by 2 */
background-color: #333; /* Black background color */
color: #fff; /* White text color */
text-align: center; /* Centered text */
border-radius: 2px; /* Rounded borders */
padding: 16px; /* Padding */
position: fixed; /* Sit on top of the screen */
z-index: 9999; /* Add a z-index if needed */
left: 10%; /* Center the snackbar */
bottom: 30px; /* 30px from the bottom */
}

#snackbar p{
color: #fff; /* White text color */
}

#snackbarTitle{
font-weight: bold; /* White text color */
}

/* Show the snackbar when clicking on a button (class added with JavaScript) */
#snackbar.show {
visibility: visible; /* Show the snackbar */
/* Add animation: Take 0.5 seconds to fade in and out the snackbar.
However, delay the fade out process for 2.5 seconds */
-webkit-animation: fadein 0.5s, fadeout 0.5s 2.5s;
animation: fadein 0.5s, fadeout 0.5s 2.5s;
}

/* Animations to fade the snackbar in and out */
@-webkit-keyframes fadein {
from {bottom: 0; opacity: 0;}
to {bottom: 30px; opacity: 1;}
}

@keyframes fadein {
from {bottom: 0; opacity: 0;}
to {bottom: 30px; opacity: 1;}
}

@-webkit-keyframes fadeout {
from {bottom: 30px; opacity: 1;}
to {bottom: 0; opacity: 0;}
}

@keyframes fadeout {
from {bottom: 30px; opacity: 1;}
to {bottom: 0; opacity: 0;}
}

 

Code for sending 

<?php
define('SERVER_API_KEY', 'XXXXXXXX:XXXXXXXX-XXXXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX-X-');

//FOR WEB ALSO TOKENS GET GENERATED. 
$tokens = ['XXXXXXXXXXXXXXXX:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX-XXXXXXXX-XXXXXXXX-XXXXXXXX',
'XXXXXXXXXXXXXXXX:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX-XXXXXXXX-XXXXXXXX-XXXXXXXX'
];

$header = [
     'Authorization: Key= ' . SERVER_API_KEY,
     'content-Type: Application/json'
];

$notification = [
      'title' => 'Testing Notification',
      'body' => 'Testing Notification Body',
      'icon' => '',
	  'sound' => 'default',
	  'click_action' => "android.intent.action.MAIN"
];

$data = [
      'title' => 'Testing Notification',
      'body' => 'Testing Notification Body',
      'icon' => '',
	  'sound' => 'default',
];

$options = [
	'priority'=> 'high'
];

$payload = [
       'registration_ids' => $tokens,
	   'notification' => $notification,
	   'data' => $data,
	   'options' => $options
];

$curl = curl_init();

curl_setopt_array($curl, array(
  CURLOPT_URL => "https://fcm.googleapis.com/fcm/send",
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_CUSTOMREQUEST => "POST",
  CURLOPT_POSTFIELDS => json_encode( $payload ),
  CURLOPT_HTTPHEADER => $header,
));

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
  echo "cURL Error #:" . $err;
} else {
  echo $response;
}
?>

 

Google Firebase Web Push – Public VAPID Key
Google Firebase Web Push – Web app configuration

 

Please note this code will show two popups in background mode – that is because Push is being sent as “Notification”. Browser is handling that and showing one on its own. And another from the backgroundHandler code in Service Worker file. For showing only the custom popup use “Data” instead of “Notification” in the Push body.

 

How to find list of all FCM parameters https://www.kolkataonweb.com/code-bank/miscellaneous/google-fcm-all-push-notification-parameters/

Playing RTMP in Browser without FLASH

Miscellaneous, PHP, , , , , ,

RTMP is used for streaming live video. With a bit of manipulation of the frontend codes it can also be used to play audio only (the below code does that)

Now the thing is playing RTMP link in browser requires Flash which is not supported by Apple and Linux based OS. The solution is to use the HLS link instead of the RTMP link. The HLS format is supported by most browsers and by Apple and Linux based OS also.

One thing to note is the MIME type of the source.

There are various Javascript libraries that can be used to build the player. Below is an example using Video.js

This internally uses the http-streaming library of Videojs. Details of the library can be found here  This library is included in video.js 7 by default. Hence in the below code  it has not been added separately

<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Live Streaming</title>
    <link href="https://vjs.zencdn.net/7.6.6/video-js.css" rel="stylesheet" />

	<!-- If you'd like to support IE8 (for Video.js versions prior to v7) -->
	<script src="https://vjs.zencdn.net/ie8/1.1.2/videojs-ie8.min.js"></script>
	<script src="https://vjs.zencdn.net/7.6.6/video.js"></script>
</head>
<body>
<audio id="player" class="video-js vjs-default-skin" height="70" width="300" controls autoplay preload="none">
    <source src="https://rtmpserverBaseUrl/path/streamName.m3u8" type="application/x-mpegURL" />
</audio>

<script>
    var player = videojs('#player');
    videojs('player').ready(function() {
       this.play();
    });
</script>

<style>
.vjs-seek-to-live-control,
.vjs-fullscreen-control,
.vjs-picture-in-picture-control
 {
   display: none;
 }
</style>

</body>
</html>

Code courtesy – https://github.com/videojs/http-streaming

Both libraries used in the code can be downloaded from here
videojs version 7.6.6
videojs-ie8.min version 1.1.2

JS library for Image Manipulation

Miscellaneous, , ,

The below links has various image processing functionality implemented using Javascript. They can be useful to implement various image processing functionality in website frontend.

www.marvinj.org/en/index.html

gitHub.com/inspirit/jsfeat
Some features of this library are

  • Basic image processing methods (grayscale, derivatives, box-blur, resample, etc.)
  • Grayscale
  • Box blur
  • Gaussian blur
  • Equalize histogram
  • Fast Corners feature detector
  • YAPE06 feature detector
  • YAPE feature detector
  • ORB feature descriptor
  • HAAR object detector
  • BBF object detector