In-App Extensions are implemented in the SAP S/4 HANA core and there is no remote connection needed to access the extension from the extended app. In Side-by-side extensions, SAP Cloud Platform acts as a PaaS where extension apps might be on other platforms or SAP applications having a connection with the cloud platform. 

In-App Extensions resides on SAP S/4 HANA Core and hence is compatible with SAP technologies only like SAP ABAP, CDS etc. On the other hand, SAP Cloud Platform can leverage open source technologies, the Side-by-side extensions has other compliant technologies in the scope which are not developed by SAP like Java or Node.JS. 

Example of Extensions – 
01: Simplifying user experience by presenting reports in a more friendly manner. Example – remove/hide fields from report, auto-populate some values for convenience
02: Substitute a process with some logic leveraging data in SAP. Example – fetch a field from system rather than asking user to populate.
03: Pre-processing or Post-processing activities performed by extensions using third party systems. 

~S