Commit Graph

243 Commits (not-set-patch)

Author SHA1 Message Date
Uku Taht 959b2d5e7a Remove ListReport conditional render 4 years ago
Adam Rutkowski 90621331c6 Make Device section components aware of (not set)
So that no extra sub-filters are possible when the unset
top item is selected.
4 years ago
Vini Brasil 061cb6ec83
Remove the ability to collapse the main graph + transition bug fix (#2627)
* Remove the ability to collpase the top graph

This commit removes the ability to collapse the top graph. The graph
collapsed whenever `metric` was falsy. I removed all related code to
that. Metric now defaults to visitors.

We want to add new items to top stats, and this commit will make it
easier to change it. Also, there's currently a bug where top stats is
randomly collapsing, which should be fixed by this commit.

* Refactor graph and top stats loading state

The graph loading state shows and hides the graph conditionally
depending on whether the data is loaded, loading or refreshing.

The current code is a bit difficult to read because its big
conditionals. This commit refactors the loading state making it easier
to read.

This commit also fixes a bug where the graph wasn't fading out when
changing metrics.
4 years ago
Adam Rutkowski 5bb53c7e35
Revert "parse date labels by local timezone instead of UTC (#2601)" (#2619)
This reverts commit 7a7076ed20.
4 years ago
RobertJoonas 7a7076ed20
parse date labels by local timezone instead of UTC (#2601) 4 years ago
RobertJoonas c4b8bf8c32
Fix metric undefined (#2589)
* extract maybeUpdateMetric function

* reset graph metric after fetching top stats

* fix bug

Also make sure we do not allow 'metric' to become undefined anymore

* keep graph hidden when empty metric saved

* reset metric in setState callback

* pass function not call
4 years ago
RobertJoonas 3999f282a5
Last updated tooltip (#2576)
* extract blinkingDot function

* position pulsating-circle with tailwind instead

* remove unused function

* extract renderStatName function

* display seconds since last realtime update

Adds a 'Last updated X seconds ago' label to the Current Visitors tooltip.

* small refactor: avoid duplication of this.props and this.state

* show the 'last updated ...' tooltip in historical

* changelog update

* use className utility function
4 years ago
David Janda b6349df475
Display pageviews in details modal for top pages when filtering by a source (#2585) 4 years ago
David Janda f949278d39
Convert list elements to button elements for stats tabs in order to enable keyboard navigation (#2526) 4 years ago
RobertJoonas 47e21121db
Realtime dashboard improvements (#2445)
* add a new realtime-update-timer module

* hook to the new 'tick' event in ListReport for auto-updates

This commit fixes the bug where all reports using the `ListReport` component did not
auto-update in realtime mode. Those reports are:

- Pages (Top / Entry / Exit)
- Locations (Countries / Regions / Cities)
- Devices (Screen Sizes / Browsers + versions / OS-s + versions)

* fetch data for ListReports only when scrolled into view

* refactor fetching data in ListReport

* refer to one source of truth for utm tags

* make the 'All' tab in Sources auto-update

* make all UTM tabs in Sources auto-update

* fetch UTM data only when scrolled into view

* auto-update Referrers with the new timer

* auto-update google search terms

* auto-update Conversions

* make countries map auto-update

* auto-update visitor-graph and top stats with new timer

* use new tick event for current visitors (in Historical)

* remove the old timer class

* update changelog

* Visual improvements to automatic realtime updates (#2532)

* minor consistency fix for text color in dark mode

* use FlipMove in goal conversions report

* use FlipMove in ListReports

* set main graph and top stats loading state correctly

* refactor isIntervalValid function

* enforce intervals are valid when set and stored

* remove duplicate data fetching on interval change

Fetching new data is handled by the `fetchGraphData` callback in `updateInterval`

* refactor updateMetric function

* make it clearer why 'metric' can be a faulty value

* extract 'query' and 'site' variables from 'this.props'

* reset interval state only when period is changed

The 'maybeRollbackInterval' function was also used to fetch data. This commit replaces
all those function calls with 'fetchGraphData' which better describes the actual behavior.
We should only worry about rolling back the interval if 'query.period' has changed.

This commit also stops the graph from flickering when it is updated in realtime.

* update names of two variables

* remove unnecessary negation

* make collapsed graph state more explicit

* consider stored invalid intervals when graph mounts

* fix not showing loading spinner regression

* remove interval state from VisitorGraph (#2540)

* Realtime prop breakdown (#2535)

* disable load more in realtime mode

* extract doFetch function

* separate fetchPropBreakdown and fetchNextPage functions

* subscribe for auto-updates in realtime

* improve readability with function name changes
4 years ago
Uku Taht a37b3433d7
Always show direct traffic in sources reports (#2531)
* Remove show_noref behaviour

Removes query param show_noref which was used from React to control
whether to show Direct / None traffic or not. The show_noref behaviour
was untested previously.

Closes #2523

* Add changelog entry

* Fix tests

* Removed files I did not mean to check in :)
4 years ago
Uku Taht 8f20bf0ab2
Add metric ID to HTML (#2518) 4 years ago
Vini Brasil 90786cb3bc
Remove intervals feature flag (#2474)
This commit removes the intervals feature flag, as this feature has
already been released to 100% of the users.
4 years ago
Vini Brasil e2563acf36
Standardise dropdown icon margins (#2473) 4 years ago
Vini Brasil ffbfb14714
Pass interval param to CSV export (#2469)
This commit adds the interval param to the CSV export client-side
request.
4 years ago
Vini Brasil 32a38fac7a
Add keyboard navigation to intervals (#2470)
* Create function to handle keybindings

* Add <i> keybinding to intervals
4 years ago
Vignesh Joglekar 497a52c10a
Add support for intervals (#1574)
Co-authored-by: Vini Brasil <vini@hey.com>
Co-authored-by: Vignesh Joglekar <hey@vigneshjoglekar.com>
4 years ago
Vini Brasil bea227d71b
Clean graph JavaScript code (#2441)
This commit lints JavaScript files, removes commented-out code, and
moves 3 constants to `graph-util.js`. This is part of the Intervals
work, in order to make the original pull request smaller and release
this feature incrementally.

Related: #1574

Co-authored-by: Vignesh Joglekar <hey@vigneshjoglekar.com>
4 years ago
Vinicius Brasil b2047d8975
Show 0% difference instead of N/A (#2401)
Closes #2397
4 years ago
Adam Rutkowski 0fa6b688af
Google APIs integration improvements (#2358)
* Make TestUtils module available in all tests

* Add macros patching the application env in tests

Unfortunately a lot of existing functionality relies on
certain application env setup. This isn't ideal because
the app config is a shared state that prevents us from
running the tests in parallel.

Those macros encapsulate setting up new env for test purposes
and make sure the changes are reverted when the test finishes.

* Allow passing request opts to HTTPClient.post/4

We need this to swap custom request building in
Google Analytics import.

* Unify errors when listing sites

* React: propagate backend error messages if available

* React: catch API errors in Search Terms component

* Propagate google API errors on referrer drilldown

* Handle verified properties errors in SC settings

* Add missing tests for SC settings controller

* Unify errors for fetching search analytics queries (list stats)

* Unify errors refreshing Google Auth Token

* Test fetch_stats/3 errors and replace Double with Mox

* Fixup makrup

* s/class/className

* Simplify Search Terms display in case of errors

* Fix warnings
4 years ago
Uku Taht 0bc762bfdc
Fix svg favicons (#2295)
* Overrides content-type for SVG favicons

* Organize favicon rendering

Make sure the placeholder icon is always requested from
/favicon/sources/placeholder

* Run prettier on site-switcher.js

* Yak Shave: upgrade Heroicons to 2.0

* Use HeroIcons instead of custom svg

* Update lib/plausible_web/plugs/favicon.ex

Co-authored-by: Adam Rutkowski <hq@mtod.org>

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Uku Taht be1b72895d Use React.ref.current() 4 years ago
Uku Taht ad39640475 Fix tooltip boundary ref 4 years ago
Uku Taht 8d8b66d266
Add boundary to tooltip (#2211) 4 years ago
Uku Taht aa7d7a35bd Revert "Add link to UTM builder (#2195)"
This reverts commit 6e7873f834.
4 years ago
Uku Taht 6e7873f834
Add link to UTM builder (#2195)
* Add link to UTM builder

* Changelog entry
4 years ago
Uku Taht 7683638b84
Embed improvements (#2148)
* Replace current tooltip with Popper.js

* Merge tooltip and title for top stats

* Format bounce rate and visit duration numbers in tooltip

* Add 'width=manual' mode for embed

* Add changelog entry

* Use helper function canMetricBeGraphed
4 years ago
Baoshuo eac4660354
Update links to documents 4 years ago
Vinicius Brasil 2d7dee7067
Fix 24-hour clock format for some locales (#2074)
This commit fixes a commit when the 24-hour clock was displayed like
`09 Uhr:00` or `06 h:00` for some locales. It keeps appending the `:00`
string but removes any non-numeric characters first.
4 years ago
Uku Taht 446b6e64b8 Change 12h time format from '12h' to '12:00' 4 years ago
Uku Taht fc63d036e2 Use empty string in favour of regex in `String.replace` 4 years ago
Uku Taht a84da82b97 Formats the graph-util file 4 years ago
Sasha Fonseca bbe3caba6a
Show time format of graphs based on browser's language (#2048)
* Show time format of graphs based on browser's language

* Use Intl.DateTimeFormat for graph time axis
4 years ago
Vinicius Brasil ba36b08fd6
Make realtime goal conversions clickable (#2035)
Closes #1739
4 years ago
Uku Taht 93b84661d1 Add tooltip to source numbers 4 years ago
Sasha Fonseca 37dddb62d9
Add tooltip to show exact number in lists (#2009)
* Add tooltip to show exact number in lists

* Update CHANGELOG
4 years ago
Vignesh Joglekar e82131b2f3
Fixes edge case on conversion metric automatic change (#1939) 5 years ago
Uku Taht eeaefc1ad0 Only show db-ip notice when relevant 5 years ago
Uku Taht f6f78cf418 Stop content shifting on page 5 years ago
Vignesh Joglekar b686d8bff9
Re-adds underline to selected top stat (#1849) 5 years ago
Vignesh Joglekar 52348f7a7c
Reworks tooltip hierarchy (#1846) 5 years ago
RobertJoonas 40275b64d4
Pageview custom dimensions (#1816)
* added custom dimension filtering tests for pages

* first filter UI in place

* pages, entry pages and exit pages can be filtered by pageview props

* added tests for expected filtering behaviour

* fix dimension filter for sources + tests

* added is_not filtering functionality

* fixed formatting

* fixed admin_test

* added (none) as filter value + is_not filter type in UI

* added prefilling applied filter values and some UI tweaks

* added fetch options

* Make prop suggestions work with `props` filter

* Fix test

* Track login state internally

* Add CHANGELOG entry

Co-authored-by: Uku Taht <uku.taht@gmail.com>
5 years ago
Vignesh Joglekar 3b97ecdc62
Adds Main Graph Metric Selection (#1364)
* First pass bringing in previous graph improvements, and comparsion context

* Swaps issue template to new issue form syntax

* Indentation update

* Indentation update?

* More indentation

* Intendation is hard

* Finalized indentation?

* Github indentation

* Missing fields

* Formatting changes

* Checkbox changes

* Uses new timeseries API, various UI improvements, descopes conversions, ToP from graphing

* Fixes Mobile UI Issues

* Improves point detection and display on hover

* Fixes & adds tests for updated main-graph API route

* Changelog

* Changes to better metric option declaration & minor UI/default fixes

* Fixes top stat tooltips showing unformatted numbers for special (non-rounded) top stats

* Formatting

* Fixes regression with dashed portion not stopping at present_index

* Removes comparison + lint

* Improves top stat active style

* Removes comparison tests

* Splits out tooltip and top stats

Still needs:
- Tests
- Potentially more cleanup

* Adds/moves tests for top stats

* Formatting

* Updates metric LS key, removes console log

* Various fixes + cleanup

* Makes tooltip position & style more consistent

* Fixes test (returns import status on both main graph & top stats)

* Fixes interaction with month dateFormatter

* Fixes edge case tooltip behavior

It was simpler than I thought :/

* Make the entire top stat clickable

* Minor UI improvements

* Fixes another tooltip visibility edge case + cleans up boolean algebra

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
5 years ago
Uku Taht fb60032a75 Fix date display 5 years ago
Uku Taht 0e908c84f6 Show weekday and year in datepicker when relevant 5 years ago
Andrea Mazzarella 3c93a2d91b
1421 contains filter (#1799)
* small refactors and adds 'contains' to modal

* supports contains filter in the backend

* moves entry and exit page under the page filter

* prettier

* updates the CHANGELOG

* undo package-lock changes

* fixes formatting for elixir

* renames unused parameter to _

* Update changelog

* Use uppercase for constants and update type/prefix lookup
5 years ago
Jordy van den Aardweg 2c25e0b468
Prevent canvas (main graph) from being selected on long press on mobile browsers (#1801)
* Prevent canvas from being selected on long press on mobile browsers

* Update CHANGELOG.md
5 years ago
Marko Saric 2ff1cdda7c
Note about double asterisks in front and back for contains (#1783) 5 years ago
Uku Taht e27734ed79
[Continued] Google Analytics import (#1753)
* Add has_imported_stats boolean to Site

* Add Google Analytics import panel to general settings

* Get GA profiles to display in import settings panel

* Add import_from_google method as entrypoint to import data

* Add imported_visitors table

* Remove conflicting code from migration

* Import visitors data into clickhouse database

* Pass another dataset to main graph for rendering in red

This adds another entry to the JSON data returned via the main graph API
called `imported_plot`, which is similar to `plot` in form but will be
completed with previously imported data.  Currently it simply returns
the values from `plot` / 2. The data is rendered in the main graph in
red without fill, and without an indicator for the present. Rationale:
imported data will not continue to grow so there is no projection
forward, only backwards.

* Hook imported GA data to dashboard timeseries plot

* Add settings option to forget imported data

* Import sources from google analytics

* Merge imported sources when queried

* Merge imported source data native data when querying sources

* Start converting metrics to atoms so they can be subqueried

This changes "visitors" and in some places "sources" to atoms. This does
not change the behaviour of the functions - the tests all pass unchanged
following this commit. This is necessary as joining subqueries requires
that the keys in `select` statements be atoms and not strings.

* Convery GA (direct) source to empty string

* Import utm campaign and utm medium from GA

* format

* Import all data types from GA into new tables

* Handle large amounts of more data more safely

* Fix some mistakes in tables

* Make GA requests in chunks of 5 queries

* Only display imported timeseries when there is no filter

* Correctly show last 30 minutes timeseries when 'realtime'

* Add with_imported key to Query struct

* Account for injected :is_not filter on sources from dashboard

* Also add tentative imported_utm_sources table

This needs a bit more work on the google import side, as GA do not
report sources and utm sources as distinct things.

* Return imported data to dashboard for rest of Sources panel

This extends the merge_imported function definition for sources to
utm_sources, utm_mediums and utm_campaigns too. This appears to be
working on the DB side but something is incomplete on the client side.

* Clear imported stats from all tables when requested

* Merge entry pages and exit pages from imported data into unfiltered dashboard view

This requires converting the `"visits"` and `"visit_duration"` metrics
to atoms so that they can be used in ecto subqueries.

* Display imported devices, browsers and OSs on dashboard

* Display imported country data on dashboard

* Add more metrics to entries/exits for modals

* make sure data is returned via API with correct keys

* Import regions and cities from GA

* Capitalize device upon import to match native data

* Leave query limits/offsets until after possibly joining with imported data

* Also import timeOnPage and pageviews for pages from GA

* imported_countries -> imported_locations

* Get timeOnPage and pageviews for pages from GA

These are needed for the pages modal, and for calculating exit rates for
exit pages.

* Add indicator to dashboard when imported data is being used

* Don't show imported data as separately line on main graph

* "bounce_rate" -> :bounce_rate, so it works in subqueries

* Drop imported browser and OS versions

These are not needed.

* Toggle displaying imported data by clicking indicator

* Parse referrers with RefInspector

- Use 'ga:fullReferrer' instead of 'ga:source'. This provides the actual
  referrer host + path, whereas 'ga:source' includes utm_mediums and
  other values when relevant.
- 'ga:fullReferror' does however include search engine names directly,
  so they are manually checked for as RefInspector won't pick up on
  these.

* Keep imported data indicator on dashboard and strikethrough when hidden

* Add unlink google button to import panel

* Rename some GA browsers and OSes to plausible versions

* Get main top pages and exit pages panels working correctly with imported data

* mix format

* Fetch time_on_pages for imported data when needed

* entry pages need to fetch bounces from GA

* "sample_percent" -> :sample_percent as only atoms can be used in subqueries

* Calculate bounce_rate for joined native and imported data for top pages modal

* Flip some query bindings around to be less misleading

* Fixup entry page modal visit durations

* mix format

* Fetch bounces and visit_duration for sources from GA

* add more source metrics used for data in modals

* Make sources modals display correct values

* imported_visitors: bounce_rate -> bounces, avg_visit_duration -> visit_duration

* Merge imported data into aggregate stats

* Reformat top graph side icons

* Ensure sample_percent is yielded from aggregate data

* filter event_props should be strings

* Hide imported data from frontend when using filter

* Fix existing tests

* fix tests

* Fix imported indicator appearing when filtering

* comma needed, lost when rebasing

* Import utm_terms and utm_content from GA

* Merge imported utm_term and utm_content

* Rename imported Countries data as Locations

* Set imported city schema field to int

* Remove utm_terms and utm_content when clearing imported

* Clean locations import from Google Analytics

- Country and region should be set to "" when GA provides "(not set)"
- City should be set to 0 for "unknown", as we cannot reliably import
  city data from GA.

* Display imported region and city in dashboard

* os -> operating_system in some parts of code

The inconsistency of using os in some places and operating_system in
others causes trouble with subqueries and joins for the native and
imported data, which would require additional logic to account for. The
simplest solution is the just use a consistent word for all uses. This
doesn't make any user-facing or database changes.

* to_atom -> to_existing_atom

* format

* "events" metric -> :events

* ignore imported data when "events" in metrics

* update "bounce_rate"

* atomise some more metrics from new city and region api

* atomise some more metrics for email handlers

* "conversion_rate" -> :conversion_rate during csv export

* Move imported data stats code to own module

* Move imported timeseries function to Stats.Imported

* Use Timex.parse to import dates from GA

* has_imported_stats -> imported_source

* "time_on_page" -> :time_on_page

* Convert imported GA data to UTC

* Clean up GA request code a bit

There was some weird logic here with two separate lists that really
ought to be together, so this merges those.

* Fail sooner if GA timezone can't be identified

* Link imported tables to site by id

* imported_utm_content -> imported_utm_contents

* Imported GA from all of time

* Reorganise GA data fetch logic

- Fetch data from the start of time (2005)
- Check whether no data was fetched, and if so, inform user and don't
  consider data to be imported.

* Clarify removal of "visits" data when it isn't in metrics

* Apply location filters from API

This makes it consistent with the sources etc which filter out 'Direct /
None' on the API side. These filters are used by both the native and
imported data handling code, which would otherwise both duplicate the
filters in their `where` clauses.

* Do not use changeset for setting site.imported_source

* Add all metrics to all dimensions

* Run GA import in the background

* Send email when GA import completes

* Add handler to insert imported data into tests and imported_browsers_factory

* Add remaining import data test factories

* Add imported location data to test

* Test main graph with imported data

* Add imported data to operating systems tests

* Add imported data to pages tests

* Add imported data to entry pages tests

* Add imported data to exit pages tests

* Add imported data to devices tests

* Add imported data to sources tests

* Add imported data to UTM tests

* Add new test module for the data import step

* Test import of sources GA data

* Test import of utm_mediums GA data

* Test import of utm_campaigns GA data

* Add tests for UTM terms

* Add tests for UTM contents

* Add test for importing pages and entry pages data from GA

* Add test for importing exit page data

* Fix module file name typo

* Add test for importing location data from GA

* Add test for importing devices data from GA

* Add test for importing browsers data from GA

* Add test for importing OS data from GA

* Paginate GA requests to download all data

* Bump clickhouse_ecto version

* Move RefInspector wrapper function into module

* Drop timezone transform on import

* Order imported by side_id then date

* More strings -> atoms

Also changes a conditional to be a bit nicer

* Remove parallelisation of data import

* Split sources and UTM sources from fetched GA data

GA has only a "source" dimension and no "UTM source" dimension. Instead
it returns these combined. The logic herein to tease these apart is:

1. "(direct)" -> it's a direct source
2. if the source is a domain -> it's a source
3. "google" -> it's from adwords; let's make this a UTM source "adwords"
4. else -> just a UTM source

* Keep prop names in queries as strings

* fix typo

* Fix import

* Insert data to clickhouse in batches

* Fix link when removing imported data

* Merge source tables

* Import hostname as well as pathname

* Record start and end time of imported data

* Track import progress

* Fix month interval with imported data

* Do not JOIN when imported date range has no overlap

* Fix time on page using exits

Co-authored-by: mcol <mcol@posteo.net>
5 years ago
Uku Taht e70fa293f2
Pr/1609 extended (#1634)
* Change `Country` to `Location` in the Add filter dropdown

This filter allows to select region and city too

* Rename country filter group to Location

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
5 years ago