Commit Graph

1095 Commits (f95e67bec89c5287b15342ab16542e6b10555f21)

Author SHA1 Message Date
Vini Brasil 9b32dace3e
Add Content-Security-Policy to favicon Plug (#2489) 4 years ago
Serafeim Papastefanos fd6ebf7598
Trim the value in case it contains newlines (#2480)
### Changes

Hello friends, I'm trying to self-host plausible on my server and have
observed that if I add a file named (for example) `secrets/LISTEN_IP` to
configure the bind IP I'll get the following error:

```
Compiling 1 file (.ex)
** (RuntimeError) Invalid LISTEN_IP '172.19.130.83
' error: :einval
    /home/serafeim/plausible/config/runtime.exs:20: (file)
```

(notice that I get the same error when configuring the DATABASE_URL or
whatever comes first)

It seems that for whatever reason the File.read! will insert a newline
at the end of the file. Notice that I have double checked the file with
vim and *it does not contain a newline at the end* or at least I don't
know *how* to make the file not have a newline at the end.

Thus I'm adding a `String.trim()` there to fix that thing; even if I am
doing something wrong and my files *do* contain a newline at the end,
adding a `String.trim()` there definite won't hurt :)

Below you'll find a checklist. For each item on the list, check one
option and delete the other.

### Tests
- [ ] Automated tests have been added
- [x] This PR does not require tests

### Changelog
- [ ] Entry has been added to changelog
- [x] This PR does not make a user-facing change

### Documentation
- [ ] [Docs](https://github.com/plausible/docs) have been updated
- [x] This change does not need a documentation update

### Dark mode
- [ ] The UI has been tested both in dark and light mode
- [x] This PR does not change the UI
4 years ago
Vini Brasil 90786cb3bc
Remove intervals feature flag (#2474)
This commit removes the intervals feature flag, as this feature has
already been released to 100% of the users.
4 years ago
Adam Rutkowski 467317f24e
Fix: application startup, event timestamps (#2479)
### Changes

This PR:
- pushes PromEx to the bottom of supervision stack to avoid Endpoint
instrumentation failure
- ensures the site cache is ready by exposing it through the health
check endpoint
- fixes event timestamps being calculated at compile time, with
regression unit and integration tests

### Tests
- [x] Automated tests have been added
- [ ] This PR does not require tests

### Changelog
- [ ] Entry has been added to changelog
- [x] This PR does not make a user-facing change

### Documentation
- [ ] [Docs](https://github.com/plausible/docs) have been updated
- [x] This change does not need a documentation update

### Dark mode
- [ ] The UI has been tested both in dark and light mode
- [x] This PR does not change the UI
4 years ago
Adam Rutkowski 356575ef78
Gatekeep ingestion pipeline (#2472)
* Update Sites.Cache

So it's now capable of refreshing most recent sites.
Refreshing a single site is no longer wanted.

* Introduce Warmer.RecentlyUpdated

This is Sites Cache warmer that runs only for
most recently updated sites every 30s.

* Validate Request creation early

* Rename RateLimiter to GateKeeper and introduce detailed policies

* Update events API tests - a provisioned site is now required

* Update events ingestion tests

* Make limits visible in CRM Sites index

* Hard-deprecate DOMAIN_BLACKLIST

* Remove unnecessary clause

* Fix typo

* Explicitly delegate Warmer.All

* GateKeeper.allwoance => GateKeeper.check

* Instrument Sites.Cache measurments

* Update send_pageview task to output response headers

* Instrument ingestion pipeline

* Credo

* Make event telemetry test a sync case

* Simplify Request.uri/hostname handling

* Use embedded schema, apply action and rely on get_field
4 years ago
ruslandoga 77cca04417
fix ownership transfer when is_selfhost=true (#2455)
* fix ownership transfer when is_selfhost=true

* add changelog entry
4 years ago
Vini Brasil 994e7d09de
Parse event URL and domain in Plausible.Ingestion.Request (#2351)
* Parse event URL in Plausible.Ingestion.Request

* Parse event domain in Plausible.Ingestion.Request

* Rework ingestion pipeline processing (#2462)

* Rework ingestion pipeline processing

So that Request can have multiple domains and
based on that each event is processed uniformly.
The build_and_buffer/1 function now returns an
accumulator with all the dropped/buffered events
for further inspection.

* Reduce function complexity

* Don't chain struct fields to check for an empty host

* Separate referrer and utm tags

* Fix up `with` clause, credo was right cc @vinibrsl

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Vini Brasil 75fcd8d611
Fix container is not a DOM Element error (#2463) 4 years ago
Vignesh Joglekar 497a52c10a
Add support for intervals (#1574)
Co-authored-by: Vini Brasil <vini@hey.com>
Co-authored-by: Vignesh Joglekar <hey@vigneshjoglekar.com>
4 years ago
Adam Rutkowski ed9e03ae14
Outline basic rate-limiting interface (#2452)
* Outline basic rate-limiting interface

* Fixup typespecs

* Address empty cache race condition on refresh (#2457)

Ref https://github.com/plausible/analytics/pull/2444/files#r1023751222

* Fix up test case: make sure the cache is refreshed
4 years ago
RobertJoonas 135471c32e
Add tagged-events script extension (#2333)
Adds a new script extension that allows tracking interactions with specific HTML elements on a website. For example - to track link clicks on one specific `<a>` element, you can tag it like this:

```html
<a href=... class="plausible-event-name=<your_event_name>">
```

And you can also tag the link with custom property names and values:

```html
<a href=... class="plausible-event-name=<your_event_name> plausible-event-<your_custom_prop>=<your_value>">
```

Tagging a link as above will send a custom event with the given name and props, if a `click` or `auxclick` browser event happens, and targets the link element.

The tracking behavior is somewhat different based on the HTML element type:  

- `<a>` 
  - triggers on `click` and `auxclick` events
  - intercepts navigation based on the same rules as `outbound-links` and `file-downloads`
- `<form>`
  - triggers on `submit` event
  - always intercepts navigation (calls `form.submit()` after preventing default and sending the Plausible event)
- other (`<img>`, `<button>`, `<span>`, `<div>`, `<h2>`, etc ...)
  - triggers on `click` and `auxclick` events
  - does not prevent default to intercept possible navigation. Simply calls Plausible with the event name and props read from the element class list.
4 years ago
Adam Rutkowski 3fe6a95696
Update sites cache implementation (#2444)
* Allow refreshing a single site cache + clear cache on prefill

* Reorganize Site.Cache tests with describe blocks

* Tidy up Cache tests

* Make sure the cache is cleared on (p)re-fill

* Allow process name customization in Cache.Warmer

* s/Cache.prefill/Cache.refresh

* Unify Cache refresh instrumentation

* Apply credo suggestion: change `with` to `case`

* Update typespecs to pass dialyzer
4 years ago
Adam Rutkowski 457a558471
Kick off sites by domain cache implementation (#2434)
* Implement sites by domain caching interface + warmer

* Add test

* Implement hit rate interface

* Add moduledocs

* Fix up typespec

* s/warmer/warmer_fn

* Extract measure_duration/2

* Fix up typespec

* Log errors and return nil on cache internal errors

* Fix up non-existing cache test

* Retrieve specific db columns when pre-filling the cache

* Reduce the subset of fields retrieved from the DB

See 63f3c6233d (r89871536)
4 years ago
Vini Brasil 9c98a3f2e8
Add API support for intervals (#2417) 4 years ago
RobertJoonas 47bf003c29
Merge pull request #2431 from plausible/fix-email-links
Fix email links
4 years ago
Robert 061d041213 small addition to spike_notification email 4 years ago
Robert f9fc0a741e use EmailView function for greeting user and change layout email signature 4 years ago
ruslandoga 0b7870dc4d
improve first launch experience for self-hosters (#2357)
* first launch

* dynamic children, wait for repo

* remove wait_for_repo and app env manipulations

* don't mention free trial in self-hosted pages

* add changelog

* assigns[:is_selfhost] -> @is_selfhost

* better changelog wording

* rm admin_user, admin_email, admin_pwd from app env

* rm DISABLE_AUTH

* redirect / to /login when not authenticated

* remove TODO

* Update lib/plausible_web/controllers/page_controller.ex

Co-authored-by: Uku Taht <Uku.taht@gmail.com>

* format

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
4 years ago
Robert 92df3b2577 do not use default layout in internal email 4 years ago
Robert 37c927287e speak on behalf of 'we' not 'I' 4 years ago
Robert d6125ffeb1 add tests for base_email template 4 years ago
Adam Rutkowski 6c7d2eb2b5
CRM: Expose rate limiting fields per site (#2429)
* Move schemas to paths corresponding to their namespace

* Include rate-limiting site schema fields

* Expose CRM RW access to site rate-limiting fields

* Remove 👾

* Add basic moduledoc placeholders
4 years ago
Robert 69f846fb07 do not use default layout in some emails 4 years ago
Robert ff16aa9df8 remove all repeated email footers 4 years ago
Robert eaf61ff170 set unsubscribe flags for unsubscribable emails 4 years ago
Robert a45580099d replace other salutation cases and make sure @user available in assigns 4 years ago
Robert 1d2a15ea28 remove user salutations by @user as done by the layout 4 years ago
Robert 31c8ebe842 add a default base_email layout to every email 4 years ago
Vinicius Brasil 042b448d30
Trace external HTTP requests (#2419)
This commit adds OTEL tracing to external HTTP requests made with
the HttpClient module. It records the elapsed time, host, path, and
response status code.
4 years ago
Uku Taht 8bdbabbdad
Enable Dialyzer and Credo in CI (#2315) 4 years ago
Conner Vieira 271704388a
Fixed a minor typo for consistency (#2420) 4 years ago
Adam Rutkowski 9364cebb4b
Fix up Cachex metrics (#2418)
* Fix cachex size metrics

* Collect Cachex hit ratio for UAs/Sessions

* Revert profiling metrics from 101e5a68
4 years ago
Robert fbe3a4e6f1 add a test and fix rendering the dashboard link 4 years ago
Marko Saric fc3be2a5b8
Update to keep the in-app footer consistent with the main site footer (#2413) 4 years ago
Adam Rutkowski 101e5a68b5
Allow Site DB lookups during ingestion phase (#2408)
* Implement FF-driven DB lookup for sites during ingestion

We like to see the impact of doing a simple postgres lookup on each
ingestion event. The percentage-based feature flag `:ingestion_pg_lookup`
must be set in order for lookups to be executed.

* Fix resolving Cachex stats metrics

* Enable PromEx on dev env
4 years ago
Vinicius Brasil 11d7cf01c9
Remove ingestion from OTEL tracing (#2409)
This commit removes the current 1% sampling on ingestion to completely
ignore ingestion traces. We are not getting enough value from those, and
we'll keep dashboard, API and background jobs traces only.
4 years ago
Marko Saric f9a2d00582
Changing the settings link in the renewal email (#2406)
Someone mentioned there's no way to cancel from the link we include in this email (https://plausible.io/billing/upgrade) so I'm changing it to the settings page instead https://plausible.io/settings. My own https://plausible.io/billing/upgrade is a bit different than normal subscribers see it so I don't actually know if there's a way to cancel on it. If there is a way there too, feel free to ignore this
4 years ago
Adam Rutkowski c604e49f4d
Update the snippet from js/plausible.js to js/script.js (#2395)
As per https://plausible.io/docs/plausible-script
4 years ago
Adam Rutkowski a44b8ff7f5
Improve test coverage (#2392)
Co-authored-by: ruslandoga <doga.ruslan@gmail.com>
4 years ago
Adam Rutkowski 6ba5e53574
Trim goals when creating and retrieving from the DB (#2382)
People are likely to enter (copy/paste) goals from external sources
which can lead to whitespace characters appended by accident.
That renders the goal unusable and hard to distinct visually.

Normally to fix up existing goals we would use a data migration,
but this should be good enough to check if the problem
with never appearing goals resurfaces.
4 years ago
Marko Saric 252cecbf25
Correct the spelling of JavaScript (#2380)
* Update README.md

* Update snippet.html.eex

* Update settings_general.html.eex

* Update site_controller_test.exs
4 years ago
Vinicius Brasil 73444ddfb6
Fix and test Google Analytics token refresh (#2381)
This commit fixes a bug where Google Analytics import tokens were not
being refreshed properly because the function was not returning the
expected tuple. Thanks to @aerosol we can nicely test this now.
4 years ago
Adam Rutkowski 8e75f2fc07
Separate GA/SC scopes (#2372)
* Use separate scopes for GA/SC integrations

* Update tests with google scope expectations
4 years ago
Adam Rutkowski d29597ed37
Fix random typos (#2375) 4 years ago
Vinicius Brasil c9945e09cf
Suggest enterprise plan when usage is greater than 10M pageviews (#2345) 4 years ago
Vinicius Brasil a86d9697ba
Clear stats_start_date after clearing GA stats (#2370)
This commit fixes a bug where users clearing one import and trying to
re-import would get invalid import date ranges. This was caused because
the stats_start_date field was not updated to nil after clearing
imported stats, as this field defines the end date of the import.
4 years ago
Vinicius Brasil b883d44697
Implement exponential backoff for GA requests (#2366)
This commit adds exponential backoff to GA request retries, increasing
the waiting time between requests.
4 years ago
Vinicius Brasil e48851a9ed
Skip rows with empty dates from GA imports (#2359) 4 years ago
Vinicius Brasil 9a61a10273
Do not report hard bounce e-mail errors to Sentry (#2361) 4 years ago
Adam Rutkowski 0fa6b688af
Google APIs integration improvements (#2358)
* Make TestUtils module available in all tests

* Add macros patching the application env in tests

Unfortunately a lot of existing functionality relies on
certain application env setup. This isn't ideal because
the app config is a shared state that prevents us from
running the tests in parallel.

Those macros encapsulate setting up new env for test purposes
and make sure the changes are reverted when the test finishes.

* Allow passing request opts to HTTPClient.post/4

We need this to swap custom request building in
Google Analytics import.

* Unify errors when listing sites

* React: propagate backend error messages if available

* React: catch API errors in Search Terms component

* Propagate google API errors on referrer drilldown

* Handle verified properties errors in SC settings

* Add missing tests for SC settings controller

* Unify errors for fetching search analytics queries (list stats)

* Unify errors refreshing Google Auth Token

* Test fetch_stats/3 errors and replace Double with Mox

* Fixup makrup

* s/class/className

* Simplify Search Terms display in case of errors

* Fix warnings
4 years ago