Commit Graph

1593 Commits (con-cache-patch-test)

Author SHA1 Message Date
Uku Taht 3ed5e9d27d
Refactor visits metric internals (#2691)
* Stop relying on implicitly selected `visits` metric

* Make the `visits` metric more explicit in imported data queries

* Rename internally used `visits` to `__internal_visits`
4 years ago
Uku Taht 531dfb114b
Refactor: Use HeadlessUI for search-select component (#2676)
* Use HeadlessUI for search select box

* Remove downshift from package.json

* More consistent API for Combobox component

* Combine toFilterType and valueWithoutPrefix into a single function

* Rename MyCombobox -> PlausibleCombobox

* Update webpack-cli

* Disable cache for build

* Revert "Disable cache for build"

This reverts commit aa130541f8.

* Disable cache for build

* Update webpack dependencies

* Remove glob from webpack config

* Webpack is required by package.json

* Require autoprefixer in postcss config

* Revert build changes

* Fix styling for dark mode
4 years ago
Uku Taht 412e8df41b
Validate property key in breakdown API endpoint (#2686)
* Validate property key in breakdown API endpoint

* Add missing properties

* Do not allow empty custom prop
4 years ago
Vini Brasil 535874be6f
Reject unknown imported cities from queries (#2675)
* Reject unknown imported cities from queries

This commit fixes a bug where the city report returned `N/A` entries.
The functions that build imported data queries were using SQL
`COALESCE`, assuming city data is `NULL` when unknown, when actually its
unknown value is `0`.

This commit addresses the problem using SQL `NULLIF` combined with the
previous `COALESCE` call. With this change both `0` and `NULL` are
treated as unknown.

Since 1cb07efe6d cities can be `NULL`, but
previously we saved `0` as unknown.

Closes #1960

* Add entry to CHANGELOG

* Ignore cyclomatic complexity Credo check
4 years ago
Vini Brasil 1cb07efe6d
Save city name when importing from GA (#2608)
This commit adds city data to imported records from Google Analytics. The
current implementation sets city to 0 because GA does not use the GeoNames
database.

Google Analytics Reporting API uses [Geographical IDs](https://developers.google.com/analytics/devguides/collection/protocol/v1/geoid)
to identify cities and countries. Plausible uses
[GeoNames](https://geonames.org/) and I couldn't find databases corelating the
two.

Fortunately, GA also returns the city name and this commit uses the city name
and the country ISO code to find the Geoname ID. To avoid making expensive ETS
searches, I created another ETS table in the Location library that uses
{country, city} as a key.

Related PR: https://github.com/plausible/location/pull/3
4 years ago
Uku Taht 8fcf4d3304
Fix 'field key does not exist' error (#2674) 4 years ago
ruslandoga e9ba60c8b4
improve transfer ownership error message (#2651)
* improve transfer ownership error message

* add changelog

* simplify

* revert changeset invitation error message

* more descriptive error message

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Adam Rutkowski 8f85b110aa
Split Clickhouse pools into Read-Only and Read/Write (dedicated to writes) (#2661)
* Configure ingest repo access/pool size

If I'm not mistaken 3 is a sane default, the only
inserts we're doing are:

  - session buffer dump
  - events buffer dump
  - GA import dump

And all are serializable within their scopes?

* Add IngestRepo

* Start IngestRepo

* Use IngestRepo for inserts

* Annotate ClickhouseRepo as read_only

So no insert* functions are expanded

* Update moduledoc

* rename alias

* Fix default env var value so it can be casted

* Use IngestRepo for migrations

* Set default ingest pool size from 3 to 5

in case conns are restarting or else...

* Ensure all Repo prometheus metrics are collected
4 years ago
Uku Taht 4494252cce
Add data-domain to tracking script (#2663) 4 years ago
Uku Taht 3b73836136
Fix unknown bind meta (#2654)
* Fix 'unknown bind meta' error

* Changelog

* Use one case instead of nested ifs

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Uku Taht 1cc5ac0b11
Fix bug with multiple ARRAY JOIN query (#2653)
* Fix bug with multiple ARRAY JOIN query

* Add changelog entry

* Remove 'bug' label from test description

* Simplify pattern match

Co-authored-by: Adam Rutkowski <hq@mtod.org>

* Rename has_join_with_table -> joins_table?

Co-authored-by: Adam Rutkowski <hq@mtod.org>

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
ruslandoga c3e736aae6
add flash ttl (#2652) 4 years ago
Vini Brasil 5159e683cc
First pass at Comparisons (#2650)
This commit adds support for comparing the actual showed period on the main graph with the previous one. This is a first pass and it's hidden under a feature flag because it's not feature complete yet as we want to support other comparison modes.
4 years ago
Adam Rutkowski 8a2df294f5
Replace empty OS/screen size/Browser with `(not set)` in UI/API/exports (#2646)
* Make Device section components aware of (not set)

So that no extra sub-filters are possible when the unset
top item is selected.

* Support '(not set)' in breakdown/filters

* Update expectations for export tests

* Add extra tests for returning/filtering by '(not set)'

* Add changelog entry

* Remove ListReport conditional render

* Prevent redundant sub-filters

* Fix filter text rendering

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
4 years ago
Marko Saric 4ea519fd11
Some changes to the "site taken" copy (#2647)
* Some changes to the "site taken" copy

* Update site.ex

* Fix tests

---------

Co-authored-by: Vinicius Brasil <vini@hey.com>
4 years ago
Adam Rutkowski 482249ffba
Better error message on site creation error (#2644)
* Ensure clear error message on site creation error

* Update colons in API error messages
4 years ago
Adam Rutkowski f678cfab25
Prevent error reports from being sent with short feedback (#2635) 4 years ago
Adam Rutkowski 8f9f032968
Delay stats deletions (#2632)
* Implement Site removal transaction

* Implement Stats removal Oban worker

* Configure site removal queue

* Call Site.Removal.run() instead of Purge.delete_site!

* Test site/stats removal

* Remove FIXME - filed a ticket

* Over-communicate lenghty deletion process to the users
4 years ago
Marko Saric 66c65e0071
Changes to the site settings danger zone (#2629)
* Changes to the site settings danger zone

The idea was to:

1. Show a transfer site ownership message to owners of the site only. This will make ownership transfer feature more discoverable. Will also help people to transfer ownership of a site rather than deleting the whole thing when that makes more sense
2. Add a note that site deletion is a delayed operation
3. Make some other tiny copy changes

What did I break? :)

* Fix markup so the long text wraps sooner

* Fix indent

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Adam Rutkowski ad12e1ef31
Show user feedback form on server errors (#2617)
* Move Endpoint errors setup to common config

* Implement naive Sentry link resolver

* Implement error report e-mail

* Delete static sentry script

* Implement user feedback form on server errors

* Re-arrange pipe

* Use Sentry.Config.dsn() where applicable

* Fix typo

* Use Map.replace/3
4 years ago
Marko Saric c1d174d0fa
Some changes to the "locked site" copy (#2615)
Some changes to be more consistent with the emails we send. Also "valid" subscription rather than "active" subscription fits better for the different cases where this screen is shown
4 years ago
RobertJoonas bd0de97521
Fix returning more pageviews with a visit property filter (#2612)
* fix subquery for sessions in base_event_query/2

As the 'sessions' table is using the CollapsingMergeTree engine, we have
to select session_id's distinctively. Otherwise we will get multiple rows
(with sign -1 and 1) as long as the background merge hasn't happened.

* update changelog

* use GROUP BY instead of SELECT DISTINCT

* remove comma
4 years ago
ruslandoga e3f095ade7
fix class name typo and include it in safelist (#2610) 4 years ago
Adam Rutkowski 1a4b65c36c
Communicate site creation edge cases to the users (#2605)
* Fingerprint DBConnection.ConnectionError in Sentry

* Check events before creating a site

* Enable sites limit screen

* Remove debugging remnant

* Fix buggy assertions

This wasn't doing what expected:

  iex(1)> Repo.exists?(Plausible.Site, domain: "foo")
  [debug] QUERY OK source="sites" db=0.6ms idle=1906.2ms
  SELECT TRUE FROM "sites" AS s0 LIMIT 1 []

* Encapsulate check to satisfy credo

* Use less technically involved error message

* Bring back e-mail to the limit error message
4 years ago
Vini Brasil 86d5098b31
Stop logging invalid_address errors from Locus (#2604)
This commit stops logging `{:invalid_address, "ip"}` errors. This
reduces noise on Sentry, that is capturing error logs. Other errors are
still logged.
4 years ago
Uku Taht 818d4404a8
Support custom events and props in mix send_pageview (#2606) 4 years ago
ruslandoga 166748dcf2
Replace Geolix with Locus (#2362)
This PR replaces geolix with locus to simplify self-hosted setup. locus can auto-update maxmind dbs which are recommended for self-hosters if they want city-level geolocation. locus is also a bit faster.

This PR also uses a test mmdb file from https://github.com/maxmind/MaxMind-DB for e2e geolocation tests without stubs.
4 years ago
Adam Rutkowski 5ba8697c31
Fingerprint DBConnection.ConnectionError in Sentry (#2597) 4 years ago
Vini Brasil cb5d8a0d25
Make warning notices responsive (#2591)
Closes #2528
4 years ago
RobertJoonas 3999f282a5
Last updated tooltip (#2576)
* extract blinkingDot function

* position pulsating-circle with tailwind instead

* remove unused function

* extract renderStatName function

* display seconds since last realtime update

Adds a 'Last updated X seconds ago' label to the Current Visitors tooltip.

* small refactor: avoid duplication of this.props and this.state

* show the 'last updated ...' tooltip in historical

* changelog update

* use className utility function
4 years ago
Adam Rutkowski 61bf624761
Sentry cleanup (#2590)
* Filter DBConnection logs for clickhouse in Sentry

Removing the never matching clause btw

* Exclude Plug.CSRFProtection.InvalidCSRFTokenError from Sentry

* Turn common Sentry captures into logged warnings

* Exclude fonts from static conf and InvalidPathError from Sentry
4 years ago
Adam Rutkowski 97a3ce72e4
Update clickhousex & fingerprint sentry errors (#2587)
* Revert "Revert "Fingerprint Clickhousex.Error exceptions in Sentry (#2578)" (#2586)"

This reverts commit a78e07236a.

* Bump clickhousex to include https://github.com/plausible/clickhousex/pull/20
4 years ago
Adam Rutkowski a78e07236a
Revert "Fingerprint Clickhousex.Error exceptions in Sentry (#2578)" (#2586)
This reverts commit 2428a2bf36.
4 years ago
Adam Rutkowski 2428a2bf36
Fingerprint Clickhousex.Error exceptions in Sentry (#2578)
* Fingerprint Clickhousex.Error exceptions in Sentry

* Update clickhousex
4 years ago
Marko Saric 934c3c323e
Some changes to the 'dashboard locked' email copy (#2584) 4 years ago
Marko Saric 5ba446e87c
Some changes to the 'over limit' email copy (#2583) 4 years ago
Adam Rutkowski 4bde884da8
Self-host Sentry user feedback script (#2579)
* Add static, minified sentry bundle

OG source: https://browser.sentry-cdn.com/5.9.1/bundle.min.js

* On 500s, load our self-hosted sentry bundle
4 years ago
RobertJoonas 13a05b4e1e
Change metric labels for realtime with goal filter (#2560)
* change metric labels for realtime with goal filter

* changelog

* fix formatting
4 years ago
Uku Taht 51fc9c0945
Refine parallel application-side joins (#2564)
* Consolidate task timeouts, incrase to 15s

* Use Task.async_stream for parallel Clickhouse queries

* Propagate Opentelemetry context to child Task process
4 years ago
Vini Brasil 4503895d0a
Fix breakdown API pagination when using event metrics (#2562)
* Fix breakdown API pagination when using event metrics

This commit fixes a bug where the subsequent breakdown API pages had
the same items as the first page. The fix sorts the underlying
ClickHouse query by timestamp, keeping the same order between requests,
as we use OFFSET/LIMIT pagination.

* Fix repeated results assertion

* Add different ORDER BY to each breakdown property
4 years ago
Adam Rutkowski 44afbaa2f9
Wrap paddle actions with DB transactions (#2558)
* Wrap paddle actions with DB transactions

* Bow to credo

* Make the paddle transactions crash when applicable

* s/changeset/details
4 years ago
Uku Taht 1785653b1e
Ignore unknown countries (#2556)
* Ignore XX and T1 countries

* Add fallback if country_code=nil

* Lookup city overrides directly in CityOverrides module

* Changelog

* Add empty moduledoc

* Remove redundant comment
4 years ago
Vini Brasil 22e2ae1844
Add OTEL tracing to breakdown functions (#2544)
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
4 years ago
Vini Brasil 10a9767912
Remove ingestion requests and Oban queries from OTEL (#2543) 4 years ago
Vini Brasil 5152e8d416
Reject events with long URIs and data URIs (#2536)
* Reject events with data URIs

* Reject events with URIs longer than 2,000 characters

* Update CHANGELOG.md
4 years ago
Vini Brasil 0e87b489f6
Return empty list when breaking down by event:page without events (#2530)
* Return empty list when breaking down by event:page without events

This commit fixes a bug with pagination where breaking down by event:page
would always return results despite pagination.

Closes #2255

* Update CHANGELOG.md
4 years ago
Uku Taht a37b3433d7
Always show direct traffic in sources reports (#2531)
* Remove show_noref behaviour

Removes query param show_noref which was used from React to control
whether to show Direct / None traffic or not. The show_noref behaviour
was untested previously.

Closes #2523

* Add changelog entry

* Fix tests

* Removed files I did not mean to check in :)
4 years ago
Vini Brasil 478e0c6990
Handle GA authentication errors (#2505) 4 years ago
Vini Brasil 9b32dace3e
Add Content-Security-Policy to favicon Plug (#2489) 4 years ago
Serafeim Papastefanos fd6ebf7598
Trim the value in case it contains newlines (#2480)
### Changes

Hello friends, I'm trying to self-host plausible on my server and have
observed that if I add a file named (for example) `secrets/LISTEN_IP` to
configure the bind IP I'll get the following error:

```
Compiling 1 file (.ex)
** (RuntimeError) Invalid LISTEN_IP '172.19.130.83
' error: :einval
    /home/serafeim/plausible/config/runtime.exs:20: (file)
```

(notice that I get the same error when configuring the DATABASE_URL or
whatever comes first)

It seems that for whatever reason the File.read! will insert a newline
at the end of the file. Notice that I have double checked the file with
vim and *it does not contain a newline at the end* or at least I don't
know *how* to make the file not have a newline at the end.

Thus I'm adding a `String.trim()` there to fix that thing; even if I am
doing something wrong and my files *do* contain a newline at the end,
adding a `String.trim()` there definite won't hurt :)

Below you'll find a checklist. For each item on the list, check one
option and delete the other.

### Tests
- [ ] Automated tests have been added
- [x] This PR does not require tests

### Changelog
- [ ] Entry has been added to changelog
- [x] This PR does not make a user-facing change

### Documentation
- [ ] [Docs](https://github.com/plausible/docs) have been updated
- [x] This change does not need a documentation update

### Dark mode
- [ ] The UI has been tested both in dark and light mode
- [x] This PR does not change the UI
4 years ago
Vini Brasil 90786cb3bc
Remove intervals feature flag (#2474)
This commit removes the intervals feature flag, as this feature has
already been released to 100% of the users.
4 years ago
Adam Rutkowski 467317f24e
Fix: application startup, event timestamps (#2479)
### Changes

This PR:
- pushes PromEx to the bottom of supervision stack to avoid Endpoint
instrumentation failure
- ensures the site cache is ready by exposing it through the health
check endpoint
- fixes event timestamps being calculated at compile time, with
regression unit and integration tests

### Tests
- [x] Automated tests have been added
- [ ] This PR does not require tests

### Changelog
- [ ] Entry has been added to changelog
- [x] This PR does not make a user-facing change

### Documentation
- [ ] [Docs](https://github.com/plausible/docs) have been updated
- [x] This change does not need a documentation update

### Dark mode
- [ ] The UI has been tested both in dark and light mode
- [x] This PR does not change the UI
4 years ago
Adam Rutkowski 356575ef78
Gatekeep ingestion pipeline (#2472)
* Update Sites.Cache

So it's now capable of refreshing most recent sites.
Refreshing a single site is no longer wanted.

* Introduce Warmer.RecentlyUpdated

This is Sites Cache warmer that runs only for
most recently updated sites every 30s.

* Validate Request creation early

* Rename RateLimiter to GateKeeper and introduce detailed policies

* Update events API tests - a provisioned site is now required

* Update events ingestion tests

* Make limits visible in CRM Sites index

* Hard-deprecate DOMAIN_BLACKLIST

* Remove unnecessary clause

* Fix typo

* Explicitly delegate Warmer.All

* GateKeeper.allwoance => GateKeeper.check

* Instrument Sites.Cache measurments

* Update send_pageview task to output response headers

* Instrument ingestion pipeline

* Credo

* Make event telemetry test a sync case

* Simplify Request.uri/hostname handling

* Use embedded schema, apply action and rely on get_field
4 years ago
ruslandoga 77cca04417
fix ownership transfer when is_selfhost=true (#2455)
* fix ownership transfer when is_selfhost=true

* add changelog entry
4 years ago
Vini Brasil 994e7d09de
Parse event URL and domain in Plausible.Ingestion.Request (#2351)
* Parse event URL in Plausible.Ingestion.Request

* Parse event domain in Plausible.Ingestion.Request

* Rework ingestion pipeline processing (#2462)

* Rework ingestion pipeline processing

So that Request can have multiple domains and
based on that each event is processed uniformly.
The build_and_buffer/1 function now returns an
accumulator with all the dropped/buffered events
for further inspection.

* Reduce function complexity

* Don't chain struct fields to check for an empty host

* Separate referrer and utm tags

* Fix up `with` clause, credo was right cc @vinibrsl

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Vini Brasil 75fcd8d611
Fix container is not a DOM Element error (#2463) 4 years ago
Vignesh Joglekar 497a52c10a
Add support for intervals (#1574)
Co-authored-by: Vini Brasil <vini@hey.com>
Co-authored-by: Vignesh Joglekar <hey@vigneshjoglekar.com>
4 years ago
Adam Rutkowski ed9e03ae14
Outline basic rate-limiting interface (#2452)
* Outline basic rate-limiting interface

* Fixup typespecs

* Address empty cache race condition on refresh (#2457)

Ref https://github.com/plausible/analytics/pull/2444/files#r1023751222

* Fix up test case: make sure the cache is refreshed
4 years ago
RobertJoonas 135471c32e
Add tagged-events script extension (#2333)
Adds a new script extension that allows tracking interactions with specific HTML elements on a website. For example - to track link clicks on one specific `<a>` element, you can tag it like this:

```html
<a href=... class="plausible-event-name=<your_event_name>">
```

And you can also tag the link with custom property names and values:

```html
<a href=... class="plausible-event-name=<your_event_name> plausible-event-<your_custom_prop>=<your_value>">
```

Tagging a link as above will send a custom event with the given name and props, if a `click` or `auxclick` browser event happens, and targets the link element.

The tracking behavior is somewhat different based on the HTML element type:  

- `<a>` 
  - triggers on `click` and `auxclick` events
  - intercepts navigation based on the same rules as `outbound-links` and `file-downloads`
- `<form>`
  - triggers on `submit` event
  - always intercepts navigation (calls `form.submit()` after preventing default and sending the Plausible event)
- other (`<img>`, `<button>`, `<span>`, `<div>`, `<h2>`, etc ...)
  - triggers on `click` and `auxclick` events
  - does not prevent default to intercept possible navigation. Simply calls Plausible with the event name and props read from the element class list.
4 years ago
Adam Rutkowski 3fe6a95696
Update sites cache implementation (#2444)
* Allow refreshing a single site cache + clear cache on prefill

* Reorganize Site.Cache tests with describe blocks

* Tidy up Cache tests

* Make sure the cache is cleared on (p)re-fill

* Allow process name customization in Cache.Warmer

* s/Cache.prefill/Cache.refresh

* Unify Cache refresh instrumentation

* Apply credo suggestion: change `with` to `case`

* Update typespecs to pass dialyzer
4 years ago
Adam Rutkowski 457a558471
Kick off sites by domain cache implementation (#2434)
* Implement sites by domain caching interface + warmer

* Add test

* Implement hit rate interface

* Add moduledocs

* Fix up typespec

* s/warmer/warmer_fn

* Extract measure_duration/2

* Fix up typespec

* Log errors and return nil on cache internal errors

* Fix up non-existing cache test

* Retrieve specific db columns when pre-filling the cache

* Reduce the subset of fields retrieved from the DB

See 63f3c6233d (r89871536)
4 years ago
Vini Brasil 9c98a3f2e8
Add API support for intervals (#2417) 4 years ago
RobertJoonas 47bf003c29
Merge pull request #2431 from plausible/fix-email-links
Fix email links
4 years ago
Robert 061d041213 small addition to spike_notification email 4 years ago
Robert f9fc0a741e use EmailView function for greeting user and change layout email signature 4 years ago
ruslandoga 0b7870dc4d
improve first launch experience for self-hosters (#2357)
* first launch

* dynamic children, wait for repo

* remove wait_for_repo and app env manipulations

* don't mention free trial in self-hosted pages

* add changelog

* assigns[:is_selfhost] -> @is_selfhost

* better changelog wording

* rm admin_user, admin_email, admin_pwd from app env

* rm DISABLE_AUTH

* redirect / to /login when not authenticated

* remove TODO

* Update lib/plausible_web/controllers/page_controller.ex

Co-authored-by: Uku Taht <Uku.taht@gmail.com>

* format

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
4 years ago
Robert 92df3b2577 do not use default layout in internal email 4 years ago
Robert 37c927287e speak on behalf of 'we' not 'I' 4 years ago
Robert d6125ffeb1 add tests for base_email template 4 years ago
Adam Rutkowski 6c7d2eb2b5
CRM: Expose rate limiting fields per site (#2429)
* Move schemas to paths corresponding to their namespace

* Include rate-limiting site schema fields

* Expose CRM RW access to site rate-limiting fields

* Remove 👾

* Add basic moduledoc placeholders
4 years ago
Robert 69f846fb07 do not use default layout in some emails 4 years ago
Robert ff16aa9df8 remove all repeated email footers 4 years ago
Robert eaf61ff170 set unsubscribe flags for unsubscribable emails 4 years ago
Robert a45580099d replace other salutation cases and make sure @user available in assigns 4 years ago
Robert 1d2a15ea28 remove user salutations by @user as done by the layout 4 years ago
Robert 31c8ebe842 add a default base_email layout to every email 4 years ago
Vinicius Brasil 042b448d30
Trace external HTTP requests (#2419)
This commit adds OTEL tracing to external HTTP requests made with
the HttpClient module. It records the elapsed time, host, path, and
response status code.
4 years ago
Uku Taht 8bdbabbdad
Enable Dialyzer and Credo in CI (#2315) 4 years ago
Conner Vieira 271704388a
Fixed a minor typo for consistency (#2420) 4 years ago
Adam Rutkowski 9364cebb4b
Fix up Cachex metrics (#2418)
* Fix cachex size metrics

* Collect Cachex hit ratio for UAs/Sessions

* Revert profiling metrics from 101e5a68
4 years ago
Robert fbe3a4e6f1 add a test and fix rendering the dashboard link 4 years ago
Marko Saric fc3be2a5b8
Update to keep the in-app footer consistent with the main site footer (#2413) 4 years ago
Adam Rutkowski 101e5a68b5
Allow Site DB lookups during ingestion phase (#2408)
* Implement FF-driven DB lookup for sites during ingestion

We like to see the impact of doing a simple postgres lookup on each
ingestion event. The percentage-based feature flag `:ingestion_pg_lookup`
must be set in order for lookups to be executed.

* Fix resolving Cachex stats metrics

* Enable PromEx on dev env
4 years ago
Vinicius Brasil 11d7cf01c9
Remove ingestion from OTEL tracing (#2409)
This commit removes the current 1% sampling on ingestion to completely
ignore ingestion traces. We are not getting enough value from those, and
we'll keep dashboard, API and background jobs traces only.
4 years ago
Marko Saric f9a2d00582
Changing the settings link in the renewal email (#2406)
Someone mentioned there's no way to cancel from the link we include in this email (https://plausible.io/billing/upgrade) so I'm changing it to the settings page instead https://plausible.io/settings. My own https://plausible.io/billing/upgrade is a bit different than normal subscribers see it so I don't actually know if there's a way to cancel on it. If there is a way there too, feel free to ignore this
4 years ago
Adam Rutkowski c604e49f4d
Update the snippet from js/plausible.js to js/script.js (#2395)
As per https://plausible.io/docs/plausible-script
4 years ago
Adam Rutkowski a44b8ff7f5
Improve test coverage (#2392)
Co-authored-by: ruslandoga <doga.ruslan@gmail.com>
4 years ago
Adam Rutkowski 6ba5e53574
Trim goals when creating and retrieving from the DB (#2382)
People are likely to enter (copy/paste) goals from external sources
which can lead to whitespace characters appended by accident.
That renders the goal unusable and hard to distinct visually.

Normally to fix up existing goals we would use a data migration,
but this should be good enough to check if the problem
with never appearing goals resurfaces.
4 years ago
Marko Saric 252cecbf25
Correct the spelling of JavaScript (#2380)
* Update README.md

* Update snippet.html.eex

* Update settings_general.html.eex

* Update site_controller_test.exs
4 years ago
Vinicius Brasil 73444ddfb6
Fix and test Google Analytics token refresh (#2381)
This commit fixes a bug where Google Analytics import tokens were not
being refreshed properly because the function was not returning the
expected tuple. Thanks to @aerosol we can nicely test this now.
4 years ago
Adam Rutkowski 8e75f2fc07
Separate GA/SC scopes (#2372)
* Use separate scopes for GA/SC integrations

* Update tests with google scope expectations
4 years ago
Adam Rutkowski d29597ed37
Fix random typos (#2375) 4 years ago
Vinicius Brasil c9945e09cf
Suggest enterprise plan when usage is greater than 10M pageviews (#2345) 4 years ago
Vinicius Brasil a86d9697ba
Clear stats_start_date after clearing GA stats (#2370)
This commit fixes a bug where users clearing one import and trying to
re-import would get invalid import date ranges. This was caused because
the stats_start_date field was not updated to nil after clearing
imported stats, as this field defines the end date of the import.
4 years ago
Vinicius Brasil b883d44697
Implement exponential backoff for GA requests (#2366)
This commit adds exponential backoff to GA request retries, increasing
the waiting time between requests.
4 years ago
Vinicius Brasil e48851a9ed
Skip rows with empty dates from GA imports (#2359) 4 years ago
Vinicius Brasil 9a61a10273
Do not report hard bounce e-mail errors to Sentry (#2361) 4 years ago
Adam Rutkowski 0fa6b688af
Google APIs integration improvements (#2358)
* Make TestUtils module available in all tests

* Add macros patching the application env in tests

Unfortunately a lot of existing functionality relies on
certain application env setup. This isn't ideal because
the app config is a shared state that prevents us from
running the tests in parallel.

Those macros encapsulate setting up new env for test purposes
and make sure the changes are reverted when the test finishes.

* Allow passing request opts to HTTPClient.post/4

We need this to swap custom request building in
Google Analytics import.

* Unify errors when listing sites

* React: propagate backend error messages if available

* React: catch API errors in Search Terms component

* Propagate google API errors on referrer drilldown

* Handle verified properties errors in SC settings

* Add missing tests for SC settings controller

* Unify errors for fetching search analytics queries (list stats)

* Unify errors refreshing Google Auth Token

* Test fetch_stats/3 errors and replace Double with Mox

* Fixup makrup

* s/class/className

* Simplify Search Terms display in case of errors

* Fix warnings
4 years ago
Vinicius Brasil 95aafb477d
Parse Google Analytics failed response before reporting to Sentry (#2354)
This commit parses failed GA responses to JSON before reporting to
Sentry. This makes Sentry's PII filtering smarter, redacting only
specific keys from the response, instead of the whole string.
4 years ago
RobertJoonas 409f1733ce
Optimize dark mode UI for shared link form (#2355)
* remove redundant empty div

* optimize shared link form for darkmode

* follow the same style in new_api_key form
4 years ago
RobertJoonas 54071fbc1b
Merge pull request #2348 from plausible/failed-ga-import-email
add debugging info to failed GA import email template
4 years ago
Robert f6593cef17 replace contact link with reply to this email 4 years ago
Vinicius Brasil 9220d0034d
OpenTelemetry (OTEL) Implementation (#2317)
This pull request improves the current OpenTelemetry implementation. Currently only 1% of the spans are sent, due to the high volume of ingestion requests to /api/event. I enabled the 1% sampling to /api/event only, recording 100% of the other traces.
4 years ago
Marko Saric a68093b0b6
Adding LinkedIn to the in-app footer (#2347) 4 years ago
Robert cc8bed9c4a add debugging info to failed GA import email template 4 years ago
Adam Rutkowski e3ca3b32db
Include tests for Captcha success/failure scenarios (#2344)
* Include tests for Captcha success/failure scenarios

* DRY
4 years ago
Adam Rutkowski 28cec9d939
Automatically decode JSON payloads from HTTP (#2329) 4 years ago
Vinicius Brasil fb05e447f9
Increase Google Analytics timeout and lower page size (#2326)
The Google Analytics report request may take some time, especially with big
imports with thousands of pages. To mitigate the issue this commit increases
the timeout to 60s and lowers the page size to 7,500 records per request.
4 years ago
Adam Rutkowski ae4ae5d0a0
Handle duplicate invitations gracefully (#2323) 4 years ago
Adam Rutkowski ec90a264b4
Improve user input validation (#2291)
* Make pagination params parsing ignore bad input

* Remove unused binding

* Don't crash on filter parse error

* Sanitize input date on internal stats API

* Revert Query module changes (ref 55645734)

* Implement simplistic input date validation in stats controller

* Mute bad request logging
4 years ago
Vinicius Brasil 8f6f860bd2
Add --host option to send_event task (#2320)
This commit adds an option to override `http://localhost:8000` in
`mix send_event`. This is useful when running the server in another port
or with remote servers such as staging.
4 years ago
Vinicius Brasil 0733efa89e
Delete stats improvements (#2318)
* Move clear stats functions to Plausible.Purge

* Delete both native and imported stats when deleting a site

This commit moves the delete site function to the Plausible.Purge
module, and fixes a bug where deleted sites could leave dangling
imported stats.

* Clear sites.stats_start_date after clearing stats

This commit fixes a bug where resetting stats left an invalid state of
the stats_start_date field, used for GA imports, for example.
4 years ago
Uku Taht aadf892a71
Increase DB query timeout for site transfers (#2309) 4 years ago
Vinicius Brasil 4a2e0af337
Override stats_start_date in CRM (#2310)
* Refactor Plausible.Sites.stats_start_date/1 function

This commit adds documentation, typespecs and better pattern matching to
the Plausible.Sites.stats_start_date/1 function.

* Use memoized stats_start_date instead of querying CH

* Prevent domain from being updated from CRM

* Add stats_start_date to CRM site form

This commit adds stats_start_date to CRM site form, and creates a
separate Ecto changeset for CRM changes.

* Alias Plausible.Site calls

Co-authored-by: Adam Rutkowski <hq@mtod.org>

* List all timezones in CRM form

* Require sites.public in CRM changeset

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Adam Rutkowski 523b114682
Allow super-admin Stats API access for locked sites (#2308) 4 years ago
Adam Rutkowski ceff6e17b2
CRM: order sites by latest. Tolerate missing owner (#2305) 4 years ago
Vinicius Brasil beabef7269
Redirect user after upgrading only when subscription is created (#2285)
* Create API to check for an active subscription

* Redirect after upgrading only when subscription is created

* Change upgrade page copy

* Add timeout notice to upgrade success page
4 years ago
Adam Rutkowski bf2b6880c5
Block Stats API for locked sites (#2302)
* Extract Sites.locked? predicate

* Lock Stats API when dashboard is locked

* Tidy tests

* Don't pollute application env from tests

* Add changelog entry

* Revert "Add changelog entry"

This reverts commit 76346074f9.
4 years ago
Adam Rutkowski 992ce0e802
Remove redundant logger from Plug (#2303) 4 years ago
Uku Taht 0bc762bfdc
Fix svg favicons (#2295)
* Overrides content-type for SVG favicons

* Organize favicon rendering

Make sure the placeholder icon is always requested from
/favicon/sources/placeholder

* Run prettier on site-switcher.js

* Yak Shave: upgrade Heroicons to 2.0

* Use HeroIcons instead of custom svg

* Update lib/plausible_web/plugs/favicon.ex

Co-authored-by: Adam Rutkowski <hq@mtod.org>

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Adam Rutkowski c8ffcc413f
Prevent sensitive information from being logged (#2300) 4 years ago
Uku Taht 4096d88171
[Sentry APP-189] Fallback to current date in all time parsing (#2277)
* Fallback to current date in all time parsing

* Add changelog
4 years ago
Uku Taht 65f0ce3061
[Sentry APP-5B] Ensure form errors can be rendered on /settings (#2278)
* Ensure form errors can be rendered on /settings

* Changelog
4 years ago
Uku Taht d104abb53d
Add fallback for favicon (#2279)
* Add fallback for favicon

* Add Favicon tests

* Changelog

* Move placeholder icon to priv folder
4 years ago
Vinicius Brasil 7489290d11
International Domain Names (IDN) Support (#2034)
* Accept letters from non-Latin alphabets in domain names

* Replace static URLs with Router functions in settings_visibility

* Beautify dashboard URL in visibility tab

* Add IDN support to CHANGELOG
4 years ago
Vinicius Brasil c07361fbce
Increase GA import request receive_timeout to 30s (#2274)
This commit increases the receive_timeout from 15s to 30s for Google
Analytics import requests. Timeouts have been reported, and I could
replay some requests that took 15-20 seconds to complete when fetching
10,000 records in a single request.
4 years ago
Uku Taht 519a1fa669
Add script to send pageview (#2272)
* Add script to send fake pageviews

* Fix option typo

* Remove dummy_event from .PHONY in Makefile
4 years ago
Vinicius Brasil 52b4ee6287
fixup! Refresh Google Analytics token before import (#2254) (#2273) 4 years ago
RobertJoonas 155e274150
Stats API: allow escaping `|` literal character with `\|` when filtering (#2266)
* add separate module for filter parsing

* add tests for filter parser

* allow escaping pipe character in filter value

* add documentation and doctests

* do not remove escape chars from wildcard values

* changelog update

* change the parse_filters/1 function argument
4 years ago
Vinicius Brasil db03cc0ac6
Fix timezone list when time changes (#2268)
This commit fixes an exception when calling `Timex.Timezone.get/2` after
there has been a timezone change. Similar issues have already been reported
in Timex (bitwaker/timex#691, bitwalker/timex#555, bitwalker/timex#625).

This skips timezones from the list when they fail to fetch, and use
`Timex.Timezone.get/2` to get the current offset using `DateTime.utc_now/0`
instead of the `NaiveDateTime` default.
4 years ago
Vinicius Brasil 1adda42a75
Refresh Google Analytics token before import (#2254)
* Capture refresh and expires from GA callback

* Pass GA refresh token to import worker

* Refresh GA token before import
4 years ago
Uku Taht e849e03058
Fix favicons (#2257) 4 years ago
RobertJoonas 80e01fedd5
do not show invoices for free subscription (#2249)
* do not show invoices for free subscription

* use refute instead of negative assert

* changed to refute in the other test too
4 years ago
Adam Rutkowski 3f7c1ce549
Aggregate DBConnection.ConnectionError in Sentry (#2260) 4 years ago
Uku Taht 3d54b88f0a
Make Finch pools lighter for self-hosting (#2250)
* Make Finch pools lighter

* Use standard http1 Finch pools
4 years ago
Vinicius Brasil 2b753cdcfd
Improve GA Imports error reporting (#2244)
* Return error tuple instead of exception when GA request fails

* Report Google Analytics failed request body to Sentry

* Improve Google Analytics function @doc

* fixup! Report Google Analytics failed request body to Sentry

* fixup! Improve Google Analytics function @doc
4 years ago
Uku Taht e16e357dd2
Fix shared link with bad auth (#2225)
* Render 404 when shared link cannot be found

* Add documentation for StatsController and shared link rendering

* Refactor shared_link/2 for more clarity

* Add changelog entry

* Use mermaid graph for sequence diagram

* Use more accurate return value in sequence diagram

* Refactor Ecto query to be more idiomatic

* Remove order dependence in test

* Restore backwards compatibility for older shared links

* Add changelog entry
4 years ago
Adam Rutkowski 30b79e5239
Inspect extra sentry payload for get_invoices/1 failure (#2248)
* Inspect extra sentry payload for get_invoices/1 failure

* Include params and invoices url in paddle trace
4 years ago
Uku Taht 669091f2ef
Ignore unknown country in imported data (#2247) 4 years ago
Adam Rutkowski df76cd2c8c
Ensure Sentry extras are maps that can be JSON-encoded (#2246)
* Ensure Sentry extras are maps that can be JSON-encoded

* Fix terminator
4 years ago
Vinicius Brasil 40182f6d6e
Manually lock outgrown enterprise accounts (#2197)
* Rename enterprise_over_limit e-mail template

* Change suggested_plan/2 function to suggest enterprise plans

* Start grace period for enterprise accounts

* Create separate module for GracePeriod schema

* Enable manual locking of enterprise accounts

* Update outgrown account alert for enterprise accounts

* Document account locking feature

* fixup! Enable manual locking of enterprise accounts
4 years ago
Adam Rutkowski 75264f8f1c
Treat non-200 HTTP responses as errors (#2243)
* Tag non-200 HTTP responses as errors

Co-authored-by: RobertJoonas <robertjoonas16@gmail.com>

* Send get_invoices/1 errors to Sentry

Co-authored-by: RobertJoonas <robertjoonas16@gmail.com>

* Keep Google API module matching non-200 responses

Co-authored-by: RobertJoonas <robertjoonas16@gmail.com>

* Make sure HTTPClient.Error.t() doesn't appear in the UI

* Unify get_invoices/1 signature

Co-authored-by: RobertJoonas <robertjoonas16@gmail.com>

Co-authored-by: RobertJoonas <robertjoonas16@gmail.com>
4 years ago
ruslandoga bc5aa84d05
enable google import for everyone (#2242) 4 years ago
Uku Taht 5d4918b66b
Allow admins to grant admin rights (#2216)
* Make sure admin can grant the admin role

* Use `site` and `current_user_role` from conn.assigns to avoid extra lookups

* Add some missing documentation

* Restrict owners in the `update_role` action

* Move @moduledoc to appropriate location

* Use Ecto.Enum to get role atoms

* Remove unused functionality

* Ensure that owners cannot lock themselves out

* Add question mark to `can_grant_role`

Co-authored-by: Adam Rutkowski <hq@mtod.org>

* Use strict short-circuit operator for booleans

Co-authored-by: Adam Rutkowski <hq@mtod.org>

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Uku Taht b239f73a6d
Ignore unknown country code (#2223)
* Ignore unknown country code

* Add changelog entry
4 years ago
RobertJoonas 0194d57b38
add upper bound to limit parameter (#2226)
* add upper bound to limit parameter

* add more strict validation for limit parameter + 2 tests

* update changelog
4 years ago
Vinicius Brasil dcd1640b1b
Sort timezone list by current offset (#2229)
This commit fixes a bug where the timezone list was sorted by the
standard offset, not considering timezone changes. The list was working
properly, but sorting was wrong.
4 years ago
Vinicius Brasil 178e34b9e7
Fix timezone input (#2222)
* Update Timex version from 3.7.7 to 3.7.8

* Generate timezone list from Tzdata

This commit fixes a bug where timezone changes weren't updating the
timezone list displayed when editing or creating a site.

Timezones were being pulled from a static list. This commit changes it
to generate the list from Tzdata, that uses a timezone database with
updated information on time changes. Additionally it adds more timezones
with aliases and links to the list.

Closes #1340

* Use timezone name from browser to recommend timezone

This commit matches the timezone name instead of offset to recommend a
timezone when creating a new site. The JavaScript Intl.DateTimeFormat
API is widely supported according to the link. In any case, if the
timezone fails to match by name, it fallbacks to the offset strategy.

https://caniuse.com/mdn-javascript_builtins_intl_datetimeformat_resolvedoptions_computed_timezone

Closes #904
4 years ago
Vinicius Brasil 119640486e
Remove parallel requests to Google Analytics (#2219) 4 years ago
Adam Rutkowski cc4d4bb8e0
Fixup error reporting (ref #2181) (#2204)
* Fixup error reporting (ref #2181)

Fixes two problems:

 - `extra` sent to Sentry is always expected to be a map
 - no `String.Chars` protocol is implemented for `Finch.Error` struct

* Use explicit paths in http client tests

* Make HTTP Client tests sync
4 years ago
Vinicius Brasil d31db86b49
List all Google Analytics views during import (#2184)
* List all Google Analytics views during import

This commit fixes a bug where different Google Analytics views with the
same name and URI were not shown. This was caused because GA views were
stored as a map, that naturally doesn't support duplicate keys.

This change updates the GA views list to display view IDs, making it
clearer to know what is being imported. The dropdown is now grouped by
website URL.

* Put Google Analytics API URLs in app env

* Add controller test to GA view list
4 years ago
Adam Rutkowski e2aa519c31
Test HTTPClient (#2185)
* Sort dependencies

* Add :bypass dependency

* Test HTTPClient

* Rename test module
4 years ago
Vinicius Brasil 263a359366
Add missing `application/` to Content-Type match (#2183) 4 years ago
Adam Rutkowski 1a294854e3
Fix improper pattern matching on Finch responses (#2181) 4 years ago
Adam Rutkowski 2275cd8e22 Respond with 400 when no auth supplied to shared link request 4 years ago
Marko Saric b9ae568bad
Fixing Innenstadt II/III to Frankfurt (#2162) 4 years ago
Vinicius Brasil e417c82a26
Perform calculations in SQL instead of app when creating sites (#2156)
This commit removes some Enum calls to rely on the database for
aggregating data. This improves performance when creating new sites,
especially if the user has multiple sites.
4 years ago
Uku Taht 7683638b84
Embed improvements (#2148)
* Replace current tooltip with Popper.js

* Merge tooltip and title for top stats

* Format bounce rate and visit duration numbers in tooltip

* Add 'width=manual' mode for embed

* Add changelog entry

* Use helper function canMetricBeGraphed
4 years ago
Vinicius Brasil f4a242ac71
Handle unexpected Google Analytics responses (#2158)
This commit handles unexpected Google Analytics responses while
importing, and logs to Sentry and Logger. This will improve
observability when things go wrong with GA imports.
4 years ago
Vinicius Brasil bc68235d04
Add fallback for GA reports without rows (#2121) 4 years ago
Uku Taht 00d82fd5b1 Increase user_agents cache size to 10k 4 years ago
Uku Taht f4db53109c Add guard to CRM subscription_plan function 4 years ago
Uku Taht 7bfd7cc219 Fix captcha HTTP request 4 years ago
Uku Taht eb99133791 Fix favicon requests 4 years ago
RobertJoonas 4f4b05335e
link to contact page instead of mailto (#2108) 4 years ago
Vinicius Brasil a10d44a0d7
Refactor event struct creation function (#2098)
* Replace Ingestion.Request headers with user_agent

* Replace generic Ingestion.Request params with specific fields

* Refactor event building function into small functions

* Move Plausible.Ingestion to Plausible.Ingestion.Event

* Add option to override event fields while building

* Rename Ingestion.Request meta to props

* Replace UTM-specific fields with generic query_params

* Remove Map.from_struct/1 call from ingestion pipeline

* Remove stash options from ingestion
4 years ago
Uku Taht ad739f603b Fix compile error 4 years ago
Uku Taht 57c758cff9 Fix favicon HTTP response handling 4 years ago
Vinicius Brasil 4d20c7ce70
Catch Google Search Console grant error (#2101)
* Remove invalid Jason.decode argument

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

* Add custom message to Google invalid grant error

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

* Test invalid_grant while refreshing Google token

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
4 years ago
RobertJoonas 8595332717
Merge pull request #2105 from renbaoshuo/update-links-to-docs
Update links to documents
4 years ago
Manu S Ajith 777c69b1d6
Migrate HTTPoison to Finch (#2054)
Signed-off-by: Manu S Ajith <neo@codingarena.in>

Signed-off-by: Manu S Ajith <neo@codingarena.in>
4 years ago
Baoshuo eac4660354
Update links to documents 4 years ago
Uku Taht 5c83ea77de Remove cache reporting to logs 4 years ago
Manu S Ajith c7a9da75b4
Enable Cachex stats (#2100)
Signed-off-by: Manu S Ajith <neo@codingarena.in>

Signed-off-by: Manu S Ajith <neo@codingarena.in>
4 years ago
Manu S Ajith c0c36646e2
Add Custom telemetry for Plausible.Event.WriteBuffer, Plausible.Event.WriteBuffer and Cachex (#2095)
* Add Custom telemetry for Plausible.Event.WriteBuffer, Plausible.Event.WriteBuffer and Cachex

Signed-off-by: Manu S Ajith <neo@codingarena.in>

* Rename telemetry.ex to avoid confusion with Phoenix Telemetry supervisor

Signed-off-by: Manu S Ajith <neo@codingarena.in>

* Remove duplicate event

Signed-off-by: Manu S Ajith <neo@codingarena.in>

Signed-off-by: Manu S Ajith <neo@codingarena.in>
4 years ago
Uku Taht a5757bc61e Allow ownership transfers 4 years ago
Vinicius Brasil 6b8ed12567
Fix Google Search Console API call (#2090)
This commit fixes a bug where fetching Google Search Console keywords
raised a FunctionClauseError. This was introduced in #2046. This commit
also adds test coverage.
4 years ago
ruslandoga ce461b5192 Auth updates 4 years ago
Vinicius Brasil d41fd68e99
Create struct for event requests (#2084)
* Create struct for saving ingestion request

* Create separate function to buffer events
4 years ago
RobertJoonas a058cf6240
added a test and changed hashmode condition (#2082) 4 years ago
Vinicius Brasil 4b9032d822
Google Analytics Import Refactor (#2046)
* Create separate module for GA HTTP requests

* Fetch GA data entirely instead of monthly

* Add buffering to GA imports

* Change positional args to maps when serializing from GA

* Create Google Analytics VCR tests
4 years ago
Vinicius Brasil b415ebe776
Fix geolocation subdivision pattern matching (#2063)
* Fix geolocation subdivision pattern matching

This commit fixes a bug where regions were not being saved. This was
caused because Geolix response was returning an additional
`:geolocation` map key. It also adds a test case for this.

Closes #2033

* Add geolocation database to .gitignore
4 years ago
Manu S Ajith ca15b605b7
Filter out noisy exception from Sentry (#2058)
Signed-off-by: Manu S Ajith <neo@codingarena.in>
4 years ago
Vinicius Brasil b5ea6ae3dc
Keep user filter when listing cities, countries, and regions stats (#2030)
This commit fixes a bug where location filters were filtering stats but
not the locations list. This was caused by a `Map.put/3` call that
overrides the user filter. This commit rollbacks 5b57143273
changes and removes the `Map.put/3` call.

Closes #1982
4 years ago
Marko Saric 2eec492f45
Remove the note about hiring (#2020) 4 years ago
Uku Taht 4a33cd0a25 Delegate to default Sentry filter 4 years ago
Uku Taht 6fbb0a24a8 Do not log Sentry.CrashError to Sentry
Stops recursive error logging to sentry
4 years ago
Sasha Fonseca db3a852f26
Fix missing 'pageviews' column from 'pages.csv' (#1988)
* Fix missing 'pageviews' column from 'pages.csv'

* Add CHANGELOG entry for Issue #1878 fix
4 years ago
Adam Rutkowski 3b82ba0e25
Upgrade to Geolix 2.0 (#1997)
* Upgrade geolix

* Remove geolix pool config

* Save unnecessary Task.async_stream roundtrip

Normally the Geolix API accepts `:where` keyword option that designates
the database to look up. In case no parameter is supplied, it'll spawn
a parallel map over all databases available. In this case we have only
one DB anyway, so there is no need for the extra instrumentation.

* Follow up on direct :geolocation lookups
4 years ago
Manu S Ajith 81f18ff0a5
Setup promex (#1999)
* Setup promex

Signed-off-by: Manu S Ajith <neo@codingarena.in>

* Cleanup promex config file

Signed-off-by: Manu S Ajith <neo@codingarena.in>
4 years ago
Uku Taht 2b8e3ea62a
Use finch in sentry client (#1996)
* Introduce Finch for Sentry integration

* Make sure the DummyAgent can be started

* No need to sanitize the dsn, finch takes care of that

* Simplify the dummy child spec

* Annotate redirects clause

* Make use of new `get_int_from_path_or_env`

* Actually use finch in Sentry config

* Configure `excluded_domains` correctly for Sentry

The way sentry is configured currently, when we get an HTTP error it
will be logged twice - once from Sentry.PlugCapture and once from
Sentry.LoggerBackend. The logger backend module does the right thing
by default but for some reason we've been overriding the config
parameter that by default stops double-counting errors. This commit
returns to the default configuration which is better.

* Default to 15s timeout

* Attempt to send twice at most

* Warn in sentry client

* Use warn level in sentry client

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Uku Taht 0553fa041b Parse geolix pool config as integers 4 years ago
Uku Taht 1cdffb098e Revert "Temporarily return 404 for favicons"
This reverts commit 8d3bdd9816.
4 years ago
Uku Taht 8c94e52f0f Adds tracing attribute when event ingest is blocked 4 years ago
Uku Taht 292a419473
Add feature flag to block event ingest (#1991)
Event ingest can be blocked using the flag `block_event_ingest`
4 years ago
Uku Taht 8d3bdd9816 Temporarily return 404 for favicons 4 years ago
Uku Taht 9617bd41b3 Revert "Add task to remove a page from the database"
This reverts commit 566d1b917a.
4 years ago
Uku Taht 58cb2f19ae Add feature flag for google analytics imports 4 years ago
Uku Taht 566d1b917a Add task to remove a page from the database 4 years ago
Marko Saric cd73df694c
We are looking for a Technical Customer Success Manager (#1977) 4 years ago
Uku Taht de17154f58 Show custom event goal form by default 4 years ago
Marko Saric e7dda24930
removing the note that we're hiring (#1958) 4 years ago
Marko Saric 2f9dbfc4a6
Changing sites into stats in this message as well (#1953)
just to make sure that it's the stats rather than actual sites that get locked
4 years ago
Uku Taht 3e5695408a
Use new Session.CacheStore in favour of Session.Store (#1934)
* Remove Session.Store in favour of Session.CacheStore

* Add CHANGELOG entry

* Use appropriate enum function
5 years ago
Kurt McAlpine b92889aaf0
Add API to retrieve site by domain (#1942) 5 years ago
Uku Taht 30b3cc1ed6 Add error state when no data is returned from GA API 5 years ago
Uku Taht 102ff1885e Trim trailing whitespace from pathname 5 years ago
Marko Saric 2ebdf61c2b
Keep the public and in-app footers consistent (#1936)
i've tried to make the two footers consistent to include the different links we were missing in the in-app footer (terms, dpa...) but also to display the new job opening for those that are logged in. i don't have a way to test it so hopefully i didn't make any mistakes
5 years ago
Marko Saric 1e2e1d83fb
Update ownership_transfer_request.html.eex (#1929) 5 years ago
Uku Taht 76edeb8276 System route 5 years ago
Uku Taht 892133d036 Parse build information 5 years ago
Uku Taht b667d65d52 Move ARG to running container instead of build container 5 years ago
RobertJoonas 225c1138b6
CRM improvements (#1903)
* lock/unlock sites, show user role, link to manage subscription

* merged remove gp and unlock sites into one single action + code style tweaks
5 years ago
Uku Taht 5b5c70b82d Add build metadata to build arg 5 years ago
Uku Taht 47fc09bacf Add version file directly to priv 5 years ago
Uku Taht da93f2aa6e Remove dead code 5 years ago
Uku Taht 7ab9a43324 Remove version.json from static file list 5 years ago
Uku Taht 32116fe2c9 Show database library in info endpoint 5 years ago
RobertJoonas 11654ddc07
Script extension additions (#1915)
* added data-include attribute to plausible.exclusions.js

* reorder extensions in filename when serving the plausible script

* fix formatting

* tweaks after review

* changelog update
5 years ago
Uku Taht cd8df23e5e Do not rely on insertion order in tests 5 years ago
Uku Taht 6f3399bb7c Fix annual notification loop 5 years ago
Uku Taht d100f1400d Show version number 5 years ago
Uku Taht e374c142f3 Fix shared link display 5 years ago
Vignesh Joglekar b7b69c6f62
Adds "invite_only" disable_registration config option (#1841)
* Adds tri-state disable_registration config

* Formatting

* Changes variable back to atom

* Changelog

* Uses atoms correctly :/

* Swaps to a more fitting value

* Formatting
5 years ago
Uku Taht eeaefc1ad0 Only show db-ip notice when relevant 5 years ago
Marko Saric 2a31f7911a
Fix Innenstadt I (#1877) 5 years ago
Uku Taht 66225a69e4 Set 30 minute session expiration 5 years ago
Uku Taht 02abbda06d Ignore nil custom prop as well 5 years ago
Uku Taht c92c548ca8 Do not error on empty string custom prop value 5 years ago
Uku Taht 660e5d37c9 Remove unused import 5 years ago
Uku Taht f18a211dcc
Ingest throughput improvement test setup (#1867)
* Add OTEL and test Cachex for sessions

* Move load test

* Start apps in the appropriate order
5 years ago
Uku Taht 26ff56d257 Use safe version for sending check stats email 5 years ago
Parker Selbert a64cc84db2
Expand oban error tracking for plugins/notifiers (#1864)
* Plugin exceptions may be silently covering up database problems such
  as failed cron job inserts.
* The previously tracked `:circuit` event is no longer emitted.
5 years ago
Uku Taht ce55dc4ae9
Retries Google Analytics API request if it returns empty rows (#1858)
* Retries Google Analytics API request if it returns empty rows

* Remove HTTPoison.Retry completely
5 years ago
RobertJoonas 199206babc
Dimensions continued (#1847)
* added the first version of dimensions extension

* finished dimensions script extension + updated tracking to use it

* script variants build
5 years ago
Uku Taht 5e415c2420 Add entry_props back in 5 years ago
Uku Taht 8fb4f3f886 Revert entry props 5 years ago
Uku Taht 428ee747c3 Hook up flags to the custom prop filter 5 years ago
RobertJoonas 40275b64d4
Pageview custom dimensions (#1816)
* added custom dimension filtering tests for pages

* first filter UI in place

* pages, entry pages and exit pages can be filtered by pageview props

* added tests for expected filtering behaviour

* fix dimension filter for sources + tests

* added is_not filtering functionality

* fixed formatting

* fixed admin_test

* added (none) as filter value + is_not filter type in UI

* added prefilling applied filter values and some UI tweaks

* added fetch options

* Make prop suggestions work with `props` filter

* Fix test

* Track login state internally

* Add CHANGELOG entry

Co-authored-by: Uku Taht <uku.taht@gmail.com>
5 years ago
Uku Taht 7c1d64458e Add fun with flags library 5 years ago
Marko Saric 13d8eecc91
add GA importer (#1840) 5 years ago
Uku Taht 9bb7f52887 Remove sensitive page from tracking 5 years ago
Uku Taht 66aceda9e9 Fix main graph selection with imported data 5 years ago
Uku Taht 05c210ad85 Enforce site limit 5 years ago
Uku Taht 3d231fc094 Redirect to correct change plan page for enterprise customers 5 years ago
Andrea Mazzarella 1128ff4bfa
Update the internal /sites api to paginate results (#1824)
* Update the internal /sites api to paginate results and adapts site-switcher to it

* Update the Changelog

* Format internal controller

* Remove the `+ Add Site` link from the site-switcher in the dashboard

* Change camel to snake case and replace imports with fully qualified calls

* Remove trailing comma from site-switcher
5 years ago
Uku Taht d2dc06ec96 Fix trial notifications 5 years ago
Vignesh Joglekar 3b97ecdc62
Adds Main Graph Metric Selection (#1364)
* First pass bringing in previous graph improvements, and comparsion context

* Swaps issue template to new issue form syntax

* Indentation update

* Indentation update?

* More indentation

* Intendation is hard

* Finalized indentation?

* Github indentation

* Missing fields

* Formatting changes

* Checkbox changes

* Uses new timeseries API, various UI improvements, descopes conversions, ToP from graphing

* Fixes Mobile UI Issues

* Improves point detection and display on hover

* Fixes & adds tests for updated main-graph API route

* Changelog

* Changes to better metric option declaration & minor UI/default fixes

* Fixes top stat tooltips showing unformatted numbers for special (non-rounded) top stats

* Formatting

* Fixes regression with dashed portion not stopping at present_index

* Removes comparison + lint

* Improves top stat active style

* Removes comparison tests

* Splits out tooltip and top stats

Still needs:
- Tests
- Potentially more cleanup

* Adds/moves tests for top stats

* Formatting

* Updates metric LS key, removes console log

* Various fixes + cleanup

* Makes tooltip position & style more consistent

* Fixes test (returns import status on both main graph & top stats)

* Fixes interaction with month dateFormatter

* Fixes edge case tooltip behavior

It was simpler than I thought :/

* Make the entire top stat clickable

* Minor UI improvements

* Fixes another tooltip visibility edge case + cleans up boolean algebra

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
5 years ago
Uku Taht 8655973069 Remove feature preview notice from GA import 5 years ago
Uku Taht f14c6c5cde Fix compile warning 5 years ago
Uku Taht 826e900c1b Make import date range look non-clickable
Because they aren't
5 years ago
Uku Taht 83c407c016
Upgrade Oban & configure Stager plugin (#1822) 5 years ago
Uku Taht 06b165eb6d Run GA import in monthly batches 5 years ago
Uku Taht 7c755d7c6d Forget has_stats 5 years ago
Uku Taht e572877cdd Use new field for all time date range as well 5 years ago
Uku Taht 5c290b1982 Use stats_start_date field on sites 5 years ago
Uku Taht 0f43f4a9df Remove unused function 5 years ago
Uku Taht 333de87ceb Add stats_start_date field 5 years ago
Uku Taht 69538e414d Fix dashboard link on Outlook 5 years ago
Marko Saric 4f45bb0480
added file downloads to the welcome email (#1811) 5 years ago
Uku Taht d40faf6ec7 More datepicker improvements 5 years ago
Uku Taht 7f58e6be4e Add Year to Date option in datepicker 5 years ago
RobertJoonas 8616dd46fb
added file-downloads script extension (#1775)
* added file-downloads script extension

* fixed the issues and made it compatible with IE

* changelog update
5 years ago
Uku Taht 8467979051 Add All Time option to datepicker 5 years ago
Uku Taht 18fd0dbe04 Always parse GA numbers as floats 5 years ago
Uku Taht ed871e7a0b Parse scientific notation in GA import 5 years ago
Uku Taht 0ab8d653be Add retry mechanism to HTTPoison 5 years ago
Uku Taht 8719865a41 Redirect user when they've already upgraded 5 years ago
Uku Taht ce649966ed Fix display 5 years ago
Uku Taht ebdd856df7 Fix tests 5 years ago
Uku Taht f00457c5d1 Change enterprise plans 5 years ago
Andrea Mazzarella 3c93a2d91b
1421 contains filter (#1799)
* small refactors and adds 'contains' to modal

* supports contains filter in the backend

* moves entry and exit page under the page filter

* prettier

* updates the CHANGELOG

* undo package-lock changes

* fixes formatting for elixir

* renames unused parameter to _

* Update changelog

* Use uppercase for constants and update type/prefix lookup
5 years ago
Uku Taht c83e038af0 Add fallback if websiteUrl is not defined 5 years ago
Marko Saric dce82833e9
changed url to ga import docs (#1796) 5 years ago
Uku Taht ae78444830 Add notice about feature preview 5 years ago
Uku Taht 4cc4e0d61b Add config flag for import testers 5 years ago
RobertJoonas e5cf800d4a
Improve stats api (#1759)
* can now query events metric when breaking down on a visit prop

* updated changelog

* fixed formatting

* fixed a bug - querying for bounce rate does not return visits anymore

* removed the changelog update
5 years ago
RobertJoonas 492f47ba1e
Crm transfer data (#1749)
* pull from master

* added query generation by struct fields

* ready, improved tests

* fixed a naming mistake
5 years ago
Uku Taht 5dc024a5cf Debug API call 5 years ago
Uku Taht a375789f52 Deal with nil when importing 5 years ago
Uku Taht 3a0ec5481b Log import issue 5 years ago
Uku Taht 8aa38e146c Fix dark mode in settings general 5 years ago
Uku Taht cd7c4bd724 Convert end_date timezone correctly 5 years ago
Uku Taht 1f32c6d8f0 Fix dark mode for GA import 5 years ago
Uku Taht ad79678ed7 Add screen for user metric 5 years ago
Uku Taht 51cd24bcaf Allow user to cancel ongoing import 5 years ago
Uku Taht 0f58d56c11 Get real start date for date range 5 years ago
Uku Taht 445b8c5d43 Show real view ID 5 years ago
Uku Taht 203f87520b
Ga import improvements (#1784)
* Do not link Google account for import

* Record start date

* Fix tests
5 years ago
Uku Taht ea947a1401 Increase HTTP timeout 5 years ago
Uku Taht f42b694f15 Send import failed email when job errors 5 years ago
Uku Taht 1fb39957a6 Import Oban error handling 5 years ago
Uku Taht 83df555f55 Remove OTEL 5 years ago
Uku Taht e1d6d119b0 Add end date to GA import 5 years ago
Uku Taht c4f7c9a7a0 Add debug loggin to GA import 5 years ago
Uku Taht dd01d79f32 Update google branding 5 years ago
Uku Taht e026148fb0 Show better error message on duplicate domain
Co-authored-by: Marko Saric <me@markosaric.com>
5 years ago
Uku Taht e27734ed79
[Continued] Google Analytics import (#1753)
* Add has_imported_stats boolean to Site

* Add Google Analytics import panel to general settings

* Get GA profiles to display in import settings panel

* Add import_from_google method as entrypoint to import data

* Add imported_visitors table

* Remove conflicting code from migration

* Import visitors data into clickhouse database

* Pass another dataset to main graph for rendering in red

This adds another entry to the JSON data returned via the main graph API
called `imported_plot`, which is similar to `plot` in form but will be
completed with previously imported data.  Currently it simply returns
the values from `plot` / 2. The data is rendered in the main graph in
red without fill, and without an indicator for the present. Rationale:
imported data will not continue to grow so there is no projection
forward, only backwards.

* Hook imported GA data to dashboard timeseries plot

* Add settings option to forget imported data

* Import sources from google analytics

* Merge imported sources when queried

* Merge imported source data native data when querying sources

* Start converting metrics to atoms so they can be subqueried

This changes "visitors" and in some places "sources" to atoms. This does
not change the behaviour of the functions - the tests all pass unchanged
following this commit. This is necessary as joining subqueries requires
that the keys in `select` statements be atoms and not strings.

* Convery GA (direct) source to empty string

* Import utm campaign and utm medium from GA

* format

* Import all data types from GA into new tables

* Handle large amounts of more data more safely

* Fix some mistakes in tables

* Make GA requests in chunks of 5 queries

* Only display imported timeseries when there is no filter

* Correctly show last 30 minutes timeseries when 'realtime'

* Add with_imported key to Query struct

* Account for injected :is_not filter on sources from dashboard

* Also add tentative imported_utm_sources table

This needs a bit more work on the google import side, as GA do not
report sources and utm sources as distinct things.

* Return imported data to dashboard for rest of Sources panel

This extends the merge_imported function definition for sources to
utm_sources, utm_mediums and utm_campaigns too. This appears to be
working on the DB side but something is incomplete on the client side.

* Clear imported stats from all tables when requested

* Merge entry pages and exit pages from imported data into unfiltered dashboard view

This requires converting the `"visits"` and `"visit_duration"` metrics
to atoms so that they can be used in ecto subqueries.

* Display imported devices, browsers and OSs on dashboard

* Display imported country data on dashboard

* Add more metrics to entries/exits for modals

* make sure data is returned via API with correct keys

* Import regions and cities from GA

* Capitalize device upon import to match native data

* Leave query limits/offsets until after possibly joining with imported data

* Also import timeOnPage and pageviews for pages from GA

* imported_countries -> imported_locations

* Get timeOnPage and pageviews for pages from GA

These are needed for the pages modal, and for calculating exit rates for
exit pages.

* Add indicator to dashboard when imported data is being used

* Don't show imported data as separately line on main graph

* "bounce_rate" -> :bounce_rate, so it works in subqueries

* Drop imported browser and OS versions

These are not needed.

* Toggle displaying imported data by clicking indicator

* Parse referrers with RefInspector

- Use 'ga:fullReferrer' instead of 'ga:source'. This provides the actual
  referrer host + path, whereas 'ga:source' includes utm_mediums and
  other values when relevant.
- 'ga:fullReferror' does however include search engine names directly,
  so they are manually checked for as RefInspector won't pick up on
  these.

* Keep imported data indicator on dashboard and strikethrough when hidden

* Add unlink google button to import panel

* Rename some GA browsers and OSes to plausible versions

* Get main top pages and exit pages panels working correctly with imported data

* mix format

* Fetch time_on_pages for imported data when needed

* entry pages need to fetch bounces from GA

* "sample_percent" -> :sample_percent as only atoms can be used in subqueries

* Calculate bounce_rate for joined native and imported data for top pages modal

* Flip some query bindings around to be less misleading

* Fixup entry page modal visit durations

* mix format

* Fetch bounces and visit_duration for sources from GA

* add more source metrics used for data in modals

* Make sources modals display correct values

* imported_visitors: bounce_rate -> bounces, avg_visit_duration -> visit_duration

* Merge imported data into aggregate stats

* Reformat top graph side icons

* Ensure sample_percent is yielded from aggregate data

* filter event_props should be strings

* Hide imported data from frontend when using filter

* Fix existing tests

* fix tests

* Fix imported indicator appearing when filtering

* comma needed, lost when rebasing

* Import utm_terms and utm_content from GA

* Merge imported utm_term and utm_content

* Rename imported Countries data as Locations

* Set imported city schema field to int

* Remove utm_terms and utm_content when clearing imported

* Clean locations import from Google Analytics

- Country and region should be set to "" when GA provides "(not set)"
- City should be set to 0 for "unknown", as we cannot reliably import
  city data from GA.

* Display imported region and city in dashboard

* os -> operating_system in some parts of code

The inconsistency of using os in some places and operating_system in
others causes trouble with subqueries and joins for the native and
imported data, which would require additional logic to account for. The
simplest solution is the just use a consistent word for all uses. This
doesn't make any user-facing or database changes.

* to_atom -> to_existing_atom

* format

* "events" metric -> :events

* ignore imported data when "events" in metrics

* update "bounce_rate"

* atomise some more metrics from new city and region api

* atomise some more metrics for email handlers

* "conversion_rate" -> :conversion_rate during csv export

* Move imported data stats code to own module

* Move imported timeseries function to Stats.Imported

* Use Timex.parse to import dates from GA

* has_imported_stats -> imported_source

* "time_on_page" -> :time_on_page

* Convert imported GA data to UTC

* Clean up GA request code a bit

There was some weird logic here with two separate lists that really
ought to be together, so this merges those.

* Fail sooner if GA timezone can't be identified

* Link imported tables to site by id

* imported_utm_content -> imported_utm_contents

* Imported GA from all of time

* Reorganise GA data fetch logic

- Fetch data from the start of time (2005)
- Check whether no data was fetched, and if so, inform user and don't
  consider data to be imported.

* Clarify removal of "visits" data when it isn't in metrics

* Apply location filters from API

This makes it consistent with the sources etc which filter out 'Direct /
None' on the API side. These filters are used by both the native and
imported data handling code, which would otherwise both duplicate the
filters in their `where` clauses.

* Do not use changeset for setting site.imported_source

* Add all metrics to all dimensions

* Run GA import in the background

* Send email when GA import completes

* Add handler to insert imported data into tests and imported_browsers_factory

* Add remaining import data test factories

* Add imported location data to test

* Test main graph with imported data

* Add imported data to operating systems tests

* Add imported data to pages tests

* Add imported data to entry pages tests

* Add imported data to exit pages tests

* Add imported data to devices tests

* Add imported data to sources tests

* Add imported data to UTM tests

* Add new test module for the data import step

* Test import of sources GA data

* Test import of utm_mediums GA data

* Test import of utm_campaigns GA data

* Add tests for UTM terms

* Add tests for UTM contents

* Add test for importing pages and entry pages data from GA

* Add test for importing exit page data

* Fix module file name typo

* Add test for importing location data from GA

* Add test for importing devices data from GA

* Add test for importing browsers data from GA

* Add test for importing OS data from GA

* Paginate GA requests to download all data

* Bump clickhouse_ecto version

* Move RefInspector wrapper function into module

* Drop timezone transform on import

* Order imported by side_id then date

* More strings -> atoms

Also changes a conditional to be a bit nicer

* Remove parallelisation of data import

* Split sources and UTM sources from fetched GA data

GA has only a "source" dimension and no "UTM source" dimension. Instead
it returns these combined. The logic herein to tease these apart is:

1. "(direct)" -> it's a direct source
2. if the source is a domain -> it's a source
3. "google" -> it's from adwords; let's make this a UTM source "adwords"
4. else -> just a UTM source

* Keep prop names in queries as strings

* fix typo

* Fix import

* Insert data to clickhouse in batches

* Fix link when removing imported data

* Merge source tables

* Import hostname as well as pathname

* Record start and end time of imported data

* Track import progress

* Fix month interval with imported data

* Do not JOIN when imported date range has no overlap

* Fix time on page using exits

Co-authored-by: mcol <mcol@posteo.net>
5 years ago
RobertJoonas b4992cedc1
Referrer spam blocklist (#1750)
* integrating blocklist library

* loads blocklist dependency from Github
5 years ago
Marko Saric 3ae2adc583
Use the correct email address for password reset (#1738) 5 years ago
Uku Taht 6bd4b73a91 Add clause for missing date 5 years ago
Uku Taht 8fa1d60ff0 Send cancellation email safely 5 years ago
RobertJoonas 775ae6228f
Streamlining support (#1723)
* can now remove user grace period from crm

* added a created_at column to users and sites in crm
5 years ago
Marko Saric c12346b6bf
Add a note to invite team members (#1715) 5 years ago
RobertJoonas 6b22806e1f
Allow admin access to locked dashboards (#1710)
* added super-admin access to locked dashboards

* fixed formatting
5 years ago
Uku Taht c2440dd96a Update tracking 5 years ago
RobertJoonas 5ef7baab70
removed a space from 6 mailto links (#1691) 5 years ago
Uku Taht b0b9b9212b Fixes password authentication issue with shared links 5 years ago
bytecode1024 d947d761a5
Fix upper limit of range A-Z (#1668)
Since the A-z range includes [, \, ], ^, _, and `, I assume this is a typo on the upper limit of the A-Z range.
5 years ago
Uku Taht 5b893bde39 Revert "Include 0 length pageviews in time_on_page calculation"
This reverts commit cee79e7875.
5 years ago
Uku Taht cee79e7875 Include 0 length pageviews in time_on_page calculation 5 years ago
Ralf Zimmermann 408d95fe09
Add custom goal props to breakdown endpoint (#1578) 5 years ago
RobertJoonas 78dbdfc092
Fix broken docs link in welcome email (#1646) 5 years ago
Uku Taht d1fe1f7e41 Cache the tracking script for 24 hours 5 years ago
Uku Taht 0505de9b42 Add site limit to enterprise upgrade plan 5 years ago
Uku Taht 52067803df Update session stats from event 5 years ago
ssendev 8077671f8a
Update session screen_size if unknown (#1610) 5 years ago
Uku Taht 0a5f6c5886 Implement wildcard filters for the API 5 years ago
Uku Taht ebb9476b56 Fix visit duration stat 5 years ago
Marko Saric 5bdd74a524
Some changes for the Netherlands (#1607) 5 years ago
Marko Saric 3e1a7a10bb
Some fixes for Austria (#1608) 5 years ago
Uku Taht 5344c477f3 Mix format 5 years ago
Uku Taht 19f1b03043 Deliver password reset email asynchronously
Fixes a potential timing attack where someone could find out
whether an email address is in our database or not.
5 years ago
Marko Saric 1f29d9d358
Change for Norway (#1618) 5 years ago
Marko Saric cc88526777
fixes for Zürich (#1612) 5 years ago
Uku Taht 6e21f28980 Access region safely 5 years ago
Uku Taht febfaa44e0 Dont error with unknown country 5 years ago
Uku Taht cd40579740 Ignore unknown country code 5 years ago
Uku Taht 81eb2b73df Notify Sentry when location code is not found 5 years ago
Uku Taht 381c7ac7bc Update domain 5 years ago
Uku Taht 23605c3d44 Add task for plans 5 years ago
Uku Taht e7f13f7d70 Remove debug statement 5 years ago
Uku Taht 35ee2fe3fb Parse bunny header 5 years ago
Uku Taht 3fb184a893 Log IP for testing 5 years ago
Uku Taht f049e7f358 Fix plan 5 years ago
Uku Taht 3f5151585a Mix format 5 years ago
Marko Saric 8673fc29e0
Couple of changes to the cities (#1599)
* Couple of changes to the cities

- list of city overrides now alphabetical
- added new overrides for switzerland and canada

* Changes for Bucharest
5 years ago
Uku Taht 9a7d1dd686 Adjust dashboard locked copy 5 years ago
Uku Taht ecc233443e Remove duplicate entry from cities map 5 years ago
Uku Taht 6f4d5b7b8c mix format 5 years ago
Marko Saric 73579054a8
Merge the two Bratislavas (#1587)
* Update settings_email_reports.html.eex

* fixed two typos

upgrage = upgrade
tranfer = transfer

* change sites to stats

changing sites to stats as couple of people mentioned it sounds like we will lock their websites so nobody can visit them

* Change from 14 to 45 times smaller

* Merge the two Bratislavas

* Some fixes to Denmark
5 years ago
Matt Colligan 8148d41e42
Add cities and regions to exported CSV data (#1485) 5 years ago
Marko Saric 1ab28875c3
change sites to stats (#1576)
* Update settings_email_reports.html.eex

* fixed two typos

upgrage = upgrade
tranfer = transfer

* change sites to stats

changing sites to stats as couple of people mentioned it sounds like we will lock their websites so nobody can visit them

* Change from 14 to 45 times smaller
5 years ago
Uku Taht 50df61cefa Add v3 plans to all_plans 5 years ago
Uku Taht f1e6baa9df Better sentry error 5 years ago
Uku Taht 0b253421ce Fix tests 5 years ago
Uku Taht 9a77644517 Better sentry message when plan is not found 5 years ago
Uku Taht 880ea45cd9 Remove cities feature flag 5 years ago
Uku Taht d4e7d27df6 Merge branch 'new-plans' 5 years ago
Uku Taht 28ab506bb8 Set start time for v3 pricing 5 years ago
Uku Taht f5fee73363 Add city overrides for mexico city 5 years ago
Uku Taht 1bef6fda52 Add stockholm override 5 years ago
Uku Taht 3fffd0c05d Add city overrides 5 years ago
Uku Taht 0bf1da09ce New plans 5 years ago
Uku Taht 303533af60 Use new testing subfolder 5 years ago
Marko Saric 7fc53124d8
Fixed two typos (#1555)
* Update settings_email_reports.html.eex

* fixed two typos

upgrage = upgrade
tranfer = transfer
5 years ago
Uku Taht b5047edc22 Truncate domain name 5 years ago
Uku Taht 711cbdfca2 Remove unused custom domain code 5 years ago
Uku Taht 051294445a Always show settings cog 5 years ago
Uku Taht f34c90e26c Show UTM term and content 5 years ago
Uku Taht 4d97583331 Use safe email functions
Fixes #1545
5 years ago
Uku Taht d48eafc436 Temporarily disable utm_term and content from csv export 5 years ago
Uku Taht 4ea0e29620 Add UTM content and term to CSV export 5 years ago
Uku Taht 48ad7485c8
PR 1393 continued (#1542)
* Add `utm_content` and `utm_term`.
Support `utm_content` and `utm_term` as requested in #515.

* Add dropdown for UTM options

* Remove utm_content and term from filter modal for now

Co-authored-by: Blender Defender <defenderblender@gmail.com>
5 years ago
RobertJoonas 231c72e8e8
add vs cloudflare link to page footer 5 years ago
Uku Taht dc8d20496c Show locked message in shared link 5 years ago
Ralf Zimmermann faf4459b15
Add "events" metrics to public breakdown endpoint (#1529) 5 years ago
Uku Taht 66575f497b Add goal type to JSON response 5 years ago
Uku Taht 97cdc4814e Refactor goals API 5 years ago
Uku Taht 1dba113e2f
[Draft] Improve location translations (#1526)
* WIP

* Use location library for search suggestions

* Remove unused code

* Remove Countries completely

* Fix tests
5 years ago
Uku Taht b49fd19934
Pull 1506 - formatted (#1527)
* adding api route PUT /api/v1/sites/goals with form fields "goal_type" and "goal_value" with supported types "event" and "page"

* arm64 docker images

* adding api route DELETE /api/v1/sites/goals/:goal_id with form param "site_id"

* revert makfile + package.json

* return statement hotfix in case site could not be found

* adding api route PUT /api/v1/sites/goals/:goal_id with form params "site_id", "goal_type", and "goal_value"

* update the goal api routes to accept event_name or page_path instead of goal_value

* cleaning goals model

* mix format

Co-authored-by: Ahmed Abbas <a.abbas@ixdc.net>
5 years ago
Uku Taht 442e401ede Use new location library 5 years ago
RobertJoonas 9022234aa6
added invoices section (#1511)
* added invoices section

* fixed formatting

* UI and code style improvements, paddle sandbox setup only on dev environment (fixed)
5 years ago
Uku Taht 139adc4056 Add more mobile browser rules 5 years ago
Uku Taht a2d0244315 Fix site locked email 5 years ago
Uku Taht 1bc9f6bfb5 Remove more dead code 5 years ago
Uku Taht 01706b7590 Remove dead code 5 years ago
Uku Taht 4d0bc61ffd Remove Twitter stuff 5 years ago
Uku Taht e7e36b994d Skip usage checking for users who are on grace period 5 years ago
Uku Taht effe56b3e4 Use new iso_codes package 5 years ago
Ru Singh 17563e99c5
UI fix: footer alignment on mobile (#1498)
* fix(ui): footer alignment on mobile

* docs(changelog): new entry for footer alignment
5 years ago
Uku Taht 7cfd9f95f1 Delete a site through the API 5 years ago
Uku Taht 07af25ffc3 Fix subscription_interval for enterprise plans 5 years ago
Uku Taht 9c07cf7ea6 Report allowance issues to Sentry 5 years ago
Uku Taht 2463296403 Fetch enterprise plan corretly 5 years ago
Uku Taht b695732521 Improve scroll behaviour on the dashboard 5 years ago
Uku Taht 37d3f2e36e Make sure that deleting an account also deletes stats 5 years ago
Uku Taht 5f6553c8d8 Remove DDG 5 years ago
Uku Taht 6fc0e7ba92 Use payment error 5 years ago
Uku Taht 7e93500834 Allow admins to access the stats API 5 years ago
Uku Taht 2bdfec1cc0
JS refactor: use generic ListReport for country report (#1487)
* Use ListReport for countries

* Fix countries tests

* Replace Browsers with ListReport

* Use Listreport for OS and screen size
5 years ago
Uku Taht dae99a94b2 Add note about embedded dashboard password protection 5 years ago
Uku Taht 6aa0d7405d Sample to 20m instead of 10m 5 years ago
Marko Saric aa5add2142
Update settings_email_reports.html.eex (#1486) 5 years ago
Uku Taht 05bf43c1be
City level location data (#1449)
* Merge branch 'plausible_master'

* Add City level details

* Add City level details

* Use ISO codes instead of geoname_id for subdivisions

* Add easier way to configure geolocation database

* Add workflow for dev branch

* Correct clickhouse migration

* Translate subdivision names

* Translate city names

* WIP

* Region and country filters

* Fix region filter

* Remove region_name when removing region filter

* Add modals for regions and cities

* Remove dead code

* WIP

* Revert "WIP"

This reverts commit 3202bf2fe9.

* Feature flag to hide cities when deployed

* Add changelog entry

* Remove unused code

* Remove unused variables

* Fix test

Co-authored-by: AymanTerra <aymanterra@yahoo.com>
5 years ago
Uku Taht 06d0d0eafa Fix over limit email reccommendation 5 years ago
Uku Taht 941a91ec22 Make better use of the index on sessions table 5 years ago
Uku Taht 218f1158d6 Show visitor counts in Search Console report
Fixes #1480
5 years ago
Uku Taht e355828465 Remove pageview filter by default 5 years ago
Uku Taht a3aede73f9 Let staging track itself 5 years ago
Uku Taht 6b55ace014 Fix error with JSON parsing 5 years ago
Uku Taht 0edf4765d4 Ignore maps in prop values 5 years ago
Uku Taht 373d05e8de Fix error with anonymous users on shared links 5 years ago
Uku Taht d3311898c5 Show message when grace period is over 5 years ago
Uku Taht e9cb8eb4e2 Remove grace period if user upgrades 5 years ago
Uku Taht ec2a4a1a65 Add [Action required] to over limit email 5 years ago
Uku Taht d547d0f996 Send email notification when sites are locked at grace period end 5 years ago
Uku Taht 29cb7462e6 Add grace period to upgrade 5 years ago
Uku Taht f88956ef3c Fix compiler warning 5 years ago
Uku Taht c4dcd87c08 Export fixes 5 years ago
Matt Colligan 7faa2f6673
CSV export output conversions and conversion rate when filtering for goal (#1464)
* Only add percentages to dashboard data when not filtering goal

* Correctly name CSV headers when exporting conversion data

* Remove percentages from tests when filtering for goal
5 years ago
Matt Colligan 3380685d40
Fix custom property total conversions value not displayed & Export custom properties (#1456)
* Fix custom property total conversions value not displayed

The custom property conversion metrics are not consistent with the other
metrics resulting in the total conversions not being displayed in the
dashboard. This fixes that.

* Export custom props of current goal when filtering dashboard for goal

This makes the CSV export also output a `prop_breakdown.csv` file which,
for the currently filtered goal, contains the conversion data for each
of its configured properties.

* Add test for goal-filtered CSV export
5 years ago
Uku Taht 7a02aae2a5 Accept custom props in expected format 5 years ago
Matt Colligan d781301886 Apply existing filter to visitors.csv in export (#1454)
Also adds a test to ensure the filter always applies.
5 years ago
Uku Taht c9f68248d8 Fix error with conversion rate 5 years ago
Matt Colligan 29e37c8342 More improvements for CSV export (#1446)
* Round time_on_page metric returned by 'pages' API

This rounds the `time_on_page` metric returned as part of the `pages`
API to the nearest second. While this shows no apparent change in the
web UI, it removes the decimal from the exported data in `pages.csv`.

* Tidy up export tests

* Round time_on_page in db query

* Add goal to CSV export tests

* Display CSV export spinner until download is ready

The mechanism used to make the page aware that the download is ready is:

- Client code sets a cookie and requests download.
- Server code handles download and removes cookie when complete.
- Client code polls every 1s to check the cookies, removing spinner when
  the export cookie is removed.
5 years ago
Uku Taht 5c02811d26 Add opentelemetry oban 5 years ago
Uku Taht 42d70bf4ea Optimization: only request top sources if notification is to be sent 5 years ago
Matt Colligan 945bf82c72
Fix conversions metrics (#1447)
f576fa2 should have updated the conversion metric names so that
`unique_conversions` and `total_conversions` are the two metrics
returned by the conversions API. This updates those so that the CSV
export outputs the correct data.
5 years ago
Uku Taht 76f6a00bec Add honeycomb 5 years ago
Matt Colligan f576fa2a2c
Improvements to CSV export (#1427)
* Add details=True to export API parameters

This makes the ZIP export add `%{"details" => "True"}` to the query's
`params` when fetching data internally for packaging in the ZIP.

This adds bounce_rate and time_on_page to the data in pages.csv, and
bounce_rate and visit_duration to sources.csv.

* Make API return data with consistent names

Some of the data types returned via the JSON or CSV API use inconsistent
naming, and some have redundant name changes (i.e. count -> visitors ->
count). This makes these all consistent and removes the redundancy.

This addresses #1426, fixes some of the CSV headers, and unifies the
JSON and CSV return data labels.

* Update changelog

* Test should use Timex.shift, not relative time

* Return full country names in CSV export

This also replaces the " character with ' in two country names, as those
are the characters used in the names, yielding a more predictable and
'correct' output.

* Fetch CSV exported data concurrently

* Use spinner to indicate when export has started

* Use 300 as default number of brekadown entries for export

Higher numbers (e.g. 1000) seem to cause clickhouse errors when there
many pages to request. It is unclear what is causing the error, as
clickhouse returns an "unknown" error code and an empty error message.
5 years ago
Uku Taht 7364733faf Ensure that question marks are encoded properly for the database 5 years ago
Uku Taht 9aa466b23f Remove unused mix task 5 years ago
Uku Taht dc080e9aa3 Remove bad EEX tag 5 years ago
Uku Taht 6f842b5603 Add better copy to locked page 5 years ago
Matt Colligan 8138a6e3a5
Start putting together means to export all aggregate data (#1387)
* Export all dashboard data in zip

This packages all data currently visible in the dashboard into
individual CSVs and downloads them together in a zip.

* Also export conversions in zip of CSVs

* Update export test with zip file response

* Add zip file download to changelog
5 years ago
Uku Taht 1b0323f5c1 Use more path helpers 5 years ago
Uku Taht a22bc91429 Add rel="noreferrer" to links from self-hosted to our site 5 years ago
Uku Taht e4bcdfa474 Revert "Get fresh search console data"
This reverts commit 3ab19afea5.
5 years ago
Uku Taht 3ab19afea5 Get fresh search console data 5 years ago
Uku Taht 9edbbe805d Update dependencies 5 years ago
Uku Taht 83b646eaea Do not send slack message on self hosted 5 years ago
Uku Taht 710f6f568a Ignores invalid query param part 5 years ago
Uku Taht 6dd9b0bb2f Do not error when bad custom prop value is supplied 5 years ago
Uku Taht 00184ce8ab Remove page from plausible tracking 5 years ago
Uku Taht f13962b387 Remove page from Plausible tracking 5 years ago
Uku Taht 4ee6257fcd Preload when needed 5 years ago
Uku Taht b9b37cdd34 Show pricing v2 when appropriate 5 years ago
Uku Taht 4eae40cd71 Apply usage optimization in background job 5 years ago
Uku Taht 26e34596c8 Allow enterprise plan users to create sites 5 years ago
Uku Taht e66e9bd1b7 Check site limit for enterprise customers 5 years ago
Uku Taht 792e534edd Optimize usage calculation for large accounts 5 years ago
Uku Taht b1594c631e Adjust API key limits for enterprise plans 5 years ago
Uku Taht cc4966a1c3 Change default API limit to 600 5 years ago
Uku Taht 6a5b383e2b Add enterprise plans 5 years ago
Uku Taht 950bd012d5 Simplify geolix database config 5 years ago
Uku Taht 2ca3baeafd Disable email verification by default 5 years ago
Uku Taht f77c37800f Use safe version of Cachex.fetch
Fixes #905
5 years ago
Vignesh Joglekar 0c982a8670
Adds list-based country report option (#1381)
* Adds list option for countries report

* Changelog

* Renames Geo to Locations

* Lint

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
5 years ago
Vignesh Joglekar 7e9d83d62e
Adds pagination for custom props & fixes an issue with errant (none) return values (#1382)
* Adds pagination for custom props, fixes errant (none) being returned in the middle of a dataset

* Formatting

* Fixes errant (none) value correctly

* Changelog

* Adds tests
5 years ago
Vignesh Joglekar f56992532f
Changes support links in self-host setup to links to GH Discussions (#1368)
* Swaps issue template to new issue form syntax

* Indentation update

* Indentation update?

* More indentation

* Intendation is hard

* Finalized indentation?

* Github indentation

* Missing fields

* Formatting changes

* Checkbox changes

* Swaps support lines in selfhost environment
5 years ago
Vignesh Joglekar b4445b1f7c
Adds various dark mode missing styles (#1367)
* Swaps issue template to new issue form syntax

* Indentation update

* Indentation update?

* More indentation

* Intendation is hard

* Finalized indentation?

* Github indentation

* Missing fields

* Formatting changes

* Checkbox changes

* Fixes various dark mode issues

* Couple more dark mode issues
5 years ago
Uku Taht e6b8a6a623 Remove unnecessary z-index 5 years ago
Felix Haase d3187e54e3
Fix z-indexes that cause menu overlay (#1365)
fixes #1352
5 years ago
Uku Taht 2e6f13ba67 Make sure ownership transfer sets trial_expiry_date 5 years ago
Uku Taht cd68761928 Fix utm_sources when filtering by event:page 5 years ago
Uku Taht 23a4f37ab4 Add 'manual' script variant 5 years ago
Uku Taht a7fd23a804 Revert "Reduce database roundtrips on critical path for authorization"
This reverts commit 71d1988ee8.
5 years ago
Uku Taht 015dcfa051 HOTFIX: add domain blacklist 5 years ago
Uku Taht bd7de59a9c Show total visitors when filtered for goal 5 years ago
Uku Taht 71d1988ee8 Reduce database roundtrips on critical path for authorization 5 years ago
Uku Taht a722d02e63 Return 400 when domain is missing 5 years ago
Uku Taht 5be38a57fa Do not return prop names when they are not used 5 years ago
Uku Taht f42f4da44b Fix how sampling is applied when filtering for a goal 5 years ago
Uku Taht 2f3bee800d Return more information in /api/event response
Fixes #1246
5 years ago
Uku Taht 27abbc8ebb Add `script` alias to the filename 5 years ago
Uku Taht 7af62e2cc4 Prevent MIME sniffing for tracker JS
Fixes #1247
5 years ago
Uku Taht a7eab3f17b Add null check to localstorage access
Fixes #1225
5 years ago
Uku Taht d93fe8197e Accept chrome extension URLs 5 years ago
Uku Taht c913d2fbb9 Fix referrer drilldown for Google
Fixes #1341
5 years ago
Ro Savage b3bc796d50
Add conversion_rate to sources api and source table (#1299)
* Add conversion_rate to sources api and source table

* Remove percentageFormatter

* Update source tests to include conversionat rate

* Add CR to detals modal

* Correct formatting with linter

* Add change log

* Add CR to Pages, Device and Countries panels
5 years ago
RobertJoonas d9a6ac4fd1
Admin - CRM shows site owner (#1345)
* Allow URLs without hostname setting it to (none)

* shows site-owner in crm

* fixed formatting

* removed extra test
5 years ago
Uku Taht 328503f236 Remove plausible tracking from invitation form 5 years ago
Uku Taht 6fe032880a Fix invitation links 5 years ago
Uku Taht 3b4db64b43 Mix format 5 years ago
Uku Taht 999a477b5d Fix get_root_domain for "(none)" 5 years ago
RobertJoonas 4742e05ab1
Allow URLs without hostname setting it to (none) (#1335) 5 years ago
Uku Taht 7e772151e3 Adjust tests 5 years ago
Uku Taht bd797c201f Mix format 5 years ago
Uku Taht 7cb148a4f1 Use only root domain in user id generation 5 years ago
Uku Taht c68b88c24b
Upgrade otp (#1313)
* update tool versions to latest

* Upgrades erlang and elixir, format project

Co-authored-by: happysalada <raphael@megzari.com>
5 years ago
Richard Willis 3f8af9a13b
Fix reading config from file (#1312) 5 years ago
Uku Taht 700a65c98a
Remove trial banner for admins & viewers (#1308)
* Start trial only when the user creates a site

* End trial when ownership is transfered
5 years ago
Uku Taht c8a1b5c73c Do not cascade user deletion to sites they do now own 5 years ago
Uku Taht ebc59313a5 Do not match trailing slash by default in page regex 5 years ago
Uku Taht 08afc2ae71 Timeout db queries in 10 seconds instead of 5 5 years ago
Uku Taht 33ba5d79b6
Memoize has_stats? (#1302)
* Memoize has_stats?

* Remove unused alias
5 years ago
Uku Taht 01412d090f Fix sampling in JOIN queries 5 years ago
Uku Taht 77d3d8695d Remove glob filters from API 5 years ago
Uku Taht 7bc40a63fe Remove unused variable 5 years ago
Uku Taht 7394a5376a Fix custom props being limited to 9 5 years ago
Uku Taht 88c2666764 Use BASE_URL in weekly report login link 5 years ago
Uku Taht a443094228 Show more than 9 goals 5 years ago
Uku Taht 0fe053e106 Fix realtime dashboard 5 years ago
Uku Taht bea26f0707 Fix custom prop breakdown when filtering with visit prop 5 years ago
Uku Taht d3ecbd4e17 Fix sampling 5 years ago
Uku Taht 63b7abad58 Disable sampling in the API 5 years ago
Uku Taht 5df15e9169 Remove IO.inspect 5 years ago
Uku Taht 43df90170b Fix compile warnings 5 years ago
Uku Taht f9ea3cda6b Fix custom props in conversions 5 years ago